We Keep Coding

JsonReaderException Unexpected state: ObjectStart [duplicate]

This question already has an answer here:
Why my dull Newtonsoft.Json deserialization code does not work? [duplicate]
(1 answer)
Closed last month.
I am trying to serialize a tree but I only need a tiny part of the data of the object (its a UI tree), so I wrote a custom converter.
The converter simply passes the reader and writer to the object
public override void WriteJson(JsonWriter writer, NavTree value, JsonSerializer serializer)
{
value.SaveAsJson(writer);
}
public override NavTree ReadJson(JsonReader reader, Type objectType, NavTree existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
NavTree tree = hasExistingValue ? existingValue : new NavTree();
tree.LoadFromJson(reader);
return tree;
}
Serialization looks like this
public void SaveAsJson(JsonWriter writer)
{
SerializableTreeItem root = new (this.GetRoot());
JObject.FromObject(root).WriteTo(writer);
}
The object appears to serialize yielding json that looks something like
"NavTree": {
"Id": "All",
"IsCategory": true,
"Children": [
{
"Id": "https://popularresistance.org/feed/",
"IsCategory": false,
"Children": []
},
{
"Id": "https://www.aljazeera.com/xml/rss/all.xml",
"IsCategory": false,
"Children": []
},
... more children
The deserialization looks like:
public void LoadFromJson(JsonReader reader)
{
SerializableTreeItem loaded =
JsonConvert.DeserializeObject<SerializableTreeItem>((string)reader.Value ?? string.Empty);
if (loaded == null) return;
if (this.GetRoot() != null)
{
this.GetRoot().Free();
TreeItem root = this.CreateItem();
root.SetMetadata(0, RootMetaDataId);
}
this.AddItem(loaded, this.GetRoot());
}
Trying to access reader.Value at the start of the function returns null.
Trying to access reader.ReadAsString() at the start results in:
Newtonsoft.Json.JsonReaderException: Unexpected state: ObjectStart. Path 'NavTree', line 669, position 14.
at Newtonsoft.Json.JsonTextReader.ReadStringValue(ReadType readType)
at Newtonsoft.Json.JsonTextReader.ReadAsString()
at Porifera.NavTree.LoadFromJson(JsonReader reader)
Line 669 is the first line of the json posted above. I never made a custom converter before so clearly I messed it up. The question is what did I do wrong? The json looks ok to me and all I really need is for the reader to deliver something and I can reconstruct the object.

You are using SerializableTreeItem as a data transfer object for NavTree:
In the field of programming a data transfer object (DTO) is an object that carries data between processes.
What you should do is to refactor your code to separate the responsibilities for converting from JSON to your DTO, and from your DTO to your NavTree.
First, modify NavTree to remove all references to JsonReader or any other JSON types:
public partial class NavTree
{
public void PopulateFromSerializableTreeItem(SerializableTreeItem loaded)
{
if (loaded == null)
return;
if (this.GetRoot() != null)
{
this.GetRoot().Free();
TreeItem root = this.CreateItem();
root.SetMetadata(0, RootMetaDataId);
}
this.AddItem(loaded, this.GetRoot());
}
public SerializableTreeItem ToSerializableTreeItem()
=> new (this.GetRoot());
}
Now, rewrite your JsonConverter<NavTree> as follows:
public class NavTreeConverter : Newtonsoft.Json.JsonConverter<NavTree>
{
public override void WriteJson(JsonWriter writer, NavTree value, JsonSerializer serializer) =>
serializer.Serialize(writer, value.ToSerializableTreeItem());
public override NavTree ReadJson(JsonReader reader, Type objectType, NavTree existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
var loaded = serializer.Deserialize<SerializableTreeItem>(reader);
// Check for null and return null? Throw an exception?
var tree = hasExistingValue ? existingValue : new NavTree();
tree.PopulateFromSerializableTreeItem(loaded);
return tree;
}
}
And you should be good to go.
Notes:
Your JsonReaderException is caused specifically by the following line:
SerializableTreeItem loaded =
JsonConvert.DeserializeObject<SerializableTreeItem>((string)reader.Value ?? string.Empty);
JsonReader.Value is the value of the current JSON token, but you are using it as if it contained the entire JSON subtree corresponding to your SerializableTreeItem. Instead, use JsonSerializer.Deserialize<T>(JsonReader) to deserialize the JSON subtree anchored by the current JSON token.
When writing, there should be no need to serialize your SerializableTreeItem to a JObject, then write the JObject. Just serialize SerializableTreeItem directly and skip the intermediate JObject representation.
By separating JSON serialization from DTO conversion, you will be able to more easily port your serialization code to System.Text.Json or any other serializer, if you eventually chose to do so.

Unexpected token: StartObject, when deserializing OneOf union type using custom converter

I am trying to deserialize the following JSON (which validates on https://jsonlint.com/):
{"pandoc-api-version":[1,22],"meta":{"title":{"t":"MetaBlocks","c":[{"t":"Para","c":[{"t":"Str","c":"Dynamic"},{"t":"Space"},{"t":"Str","c":"Language"},{"t":"Space"},{"t":"Str","c":"Runtime"}]},{"t":"Para","c":[]}]}},"blocks":[{"t":"Para","c":[{"t":"Strong","c":[{"t":"Str","c":"Bill"},{"t":"Space"},{"t":"Str","c":"Chiles"},{"t":"Space"},{"t":"Str","c":"and"},{"t":"Space"},{"t":"Str","c":"Alex"},{"t":"Space"},{"t":"Str","c":"Turner"}]}]},{"t":"Para","c":[{"t":"Emph","c":[{"t":"Strong","c":[{"t":"Str","c":"Reading"},{"t":"Space"},{"t":"Str","c":"this"},{"t":"Space"},{"t":"Str","c":"Document:"}]}]}]}]}
into the following classes:
internal record TagContent(string T, OneOf<TagContent[], string>? C);
internal class RawPandoc {
[JsonProperty] public int[] PandocApiVersion = default!;
[JsonProperty] public Dictionary<string, TagContent> Meta = default!;
[JsonProperty] public TagContent[] Blocks = default!;
}
using the following code:
var settings = new JsonSerializerSettings {
ContractResolver = new DefaultContractResolver { NamingStrategy = new KebabCaseNamingStrategy() },
Converters = new JsonConverter[] { new OneOfJsonConverter() }
};
var pandoc = JsonConvert.DeserializeObject<RawPandoc>(s, settings);
and I get the following error:
Unexpected token when deserializing object: StartObject. Path 'meta.title.c[0]', line 1, position 69.
How can I resolve this?
For completeness, here is the current and incomplete code for OneOfJsonConverter. OneOf is a library for union types in C#:
using OneOf;
namespace PandocFilters {
public class OneOfJsonConverter : JsonConverter {
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer) {
if (value is IOneOf of) {
value = of.Value;
}
serializer.Serialize(writer, value);
}
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer) {
if (reader.Value is null) { return null; }
// TODO not implemented yet
return reader.Value;
}
public override bool CanConvert(Type objectType) => objectType.UnderlyingIfNullable().GetInterfaces().Contains(typeof(IOneOf));
}
}

Problem is you are not advancing the reader in your ReadJson implementation. You declared your converter can handle IOneOf objects, and so JSON.NET expects your converter to actually read and handle it, however it does nothing as of now. So ReadJson is called (at the start of first array in json which should be deserialized to OneOf), and then after it returns - reader position is still where it was before (at start of array), which is not what JSON.NET expects. Then it fails trying to continue reading next object, because its assumptions are violated. So, just implement ReadJson, and meanwhile you can advance a reader for example like that:
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer) {
// advance reader as expected
var eitherStringOrArray = JObject.ReadFrom(reader);
return reader.Value;
}

Newtonsoft.Json v11 - DeserializeObject throws JsonReaderException

We have upgraded Newtonsoft.Json from 10.0.3 to 11.0.1 and code that has been working previously does not work anymore.
We have following JSON:
[{"mail-type":"registration","response":"250 OK id=1UjnNr-gdf-C0 ","invoice-id":,"email":"testuser08#test.com"}]
and we call following method:
var events = JsonConvert.DeserializeObject<Event[]>(jsonEvents);
This worked fine on 10.0.3, but does not on 11.0.1. On this version following exception is thrown:
Exception thrown: 'Newtonsoft.Json.JsonReaderException' in
Newtonsoft.Json.dll
Additional information: An undefined token is not a valid
System.Nullable`1[System.Int64]. Path '[0].invoice-id', line 1.
I tried to pass following options to DeserializeObject
var settings = new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore
};
But still the same error. What changes have to be done to make it work on 11.0.1. I am affraid we cannot accomodate JSON output as this comes from third party API.

Your JSON sample is not well-formed. If I upload your JSON to https://jsonlint.com/ then the following error is generated:
Error: Parse error on line 4:
...0 ", "invoice-id": , "email": "testuse
----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got ','
The line in question is as follows:
"invoice-id": ,
According to the JSON standard there needs to be a value between the : and the ,. But, why did this work in Json.NET 10.0? Apparently it was a bug, which was fixed. According to the 11.0.1 release notes:
Fix - Fixed not erroring when reading undefined for nullable long
So if we assume that your Event type looks like this:
public partial class Event
{
[JsonProperty("invoice-id")]
public long? InvoiceId { get; set; }
// Other properties as required
}
Then in 10.0 your JSON could be deserialized successfully using this type, but in 11.0 it cannot. If, however, we change InvoiceId to be an int?:
public partial class Event
{
[JsonProperty("invoice-id")]
public int? InvoiceId { get; set; }
// Other properties as required
}
It fails in both versions. Thus the fix appears to have been to handle int? and long? consistently.
Ideally, you should ask whoever sent you such JSON to fix it so that it is well-formed as defined by http://www.json.org/ and RFC 8259. If you nevertheless need to parse such JSON in the same manner as Json.NET 10.0, you could introduce TolerantNullableLongConverter as follows:
public class TolerantNullableLongConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(long?);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
if (reader.TokenType == JsonToken.Undefined)
return null;
if (reader.Value is long)
return reader.Value;
// string or int or decimal or ...
return serializer.Deserialize<long>(reader);
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
And apply it to your type as follows:
public partial class Event
{
[JsonProperty("invoice-id")]
[JsonConverter(typeof(TolerantNullableLongConverter))]
public long? InvoiceId { get; set; }
// Other properties as required
}

You can implement workaround with custom converter:
internal class NullableLongFixupConverter : JsonConverter {
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
writer.WriteValue(value);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
// if token undefined - return null
if (reader.TokenType == JsonToken.Undefined)
return null;
// otherwise - value
return (long?) reader.Value;
}
public override bool CanConvert(Type objectType) {
return objectType == typeof(long?);
}
}
Then you can either decorate properties with it:
class Event {
[JsonProperty("invoice-id")]
[JsonConverter(typeof(NullableLongFixupConverter))]
public long? InvoiceId { get; set; }
}
or register globally (will be invoked only for properties of type long?):
JsonConvert.DefaultSettings = () =>
{
var s = new JsonSerializerSettings();
s.Converters.Add(new NullableLongFixupConverter());
return s;
};

C# Reference earlier JSON element - Newtonsoft

I'm serializing an object which has a List<X> and a Dictionary<X, Y>.
The List<X> is serializing without any issues.
To serialize the dictionary however, I've implemented a custom converter.
class XYDictionaryConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(Dictionary<X, Y>).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JObject jObject = JObject.Load(reader);
Dictionary<X, Y> dictionary = new Dictionary<X, Y>();
foreach (JProperty property in jObject.Properties())
{
X key = //Get Y using property.Name
if (key != null)
{
dictionary.Add(key, property.Value.ToObject<Y>());
}
}
return dictionary;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
IDictionary<X, Y> dictionary = (IDictionary<X, Y>)value;
JObject jObject = new JObject();
foreach (var kvp in dictionary)
{
jObject.Add(kvp.Key.name, kvp.Value == null ? null : JToken.FromObject(kvp.Value));
}
jObject.WriteTo(writer);
}
}
The problem is that I have no idea how to reference the List<X> within the converter. Assuming the JSON is deserialized linearly, the List<X> should have been deserialized by this point, but as the DeserializeObject() method hasn't finished yet, I don't have any reference to it.
How would I go about solving this?

My solution in the end was to change the way I stored 'X'.
Instead of storing the keys of the dictionary as keys in the json, I stored each keyvaluepair as an array item.
Each item then is: { "key": { "$ref" : 5 } "value": { "$ref" : 10 } }
For writing, this was just a matter of using JArray, JObjectand JToken.FromObject (making sure to pass in the serializer.
For reading, jObject.GetValue("Key").ToObject<X>(serializer) worked brilliantly.
Hope this helps someone.

DataContractJsonSerializer throws InvalidCastException for members of varying types

There is a piece of string I'd like to deserialize with DataContractJsonSerializer. There is a member in the JSON string that may be an object sometimes or a string other times. Like so:
{ a: 1, b: 2, c: "c" }
or
{ a: 1, b: 2, c: {c1: 3, c2: 4} }
I know the structure of the object in the second case (let's call that class Inner), but how can DataContractJsonSerializer convert an object of two possible types?
I tried making an explicit and implicit cast operator to convert from string to Inner, but it isn't getting hit. An InvalidCastException keeps on getting thrown.

I had a similar situation, where an API was returning either an array of objects, or just the single object not in an array if there was only one result. I was completely unable to get the DataContractJsonSerializer to do this. I ended up having to switch to the JSON.NET library and fiddle with JsonConverters a bit to get it working.
In your DataContract, declare 'c' as type Inner.
[DataMember(Name = "c")]
public Inner C { get; set; }
Then write a JsonConverter to examine the token type and do the right thing when the deserializer attempts to deserialize an Inner.
internal class StringOrInnerConverter : JsonConverter {
public override bool CanConvert(Type objectType) {
return objectType == typeof(Inner);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
var ser = new JsonSerializer();
if (reader.TokenType == JsonToken.StartObject) {
var inn = ser.Deserialize<Inner>(reader);
return inn;
} else if (reader.TokenType == JsonToken.String) {
var str = ser.Deserialize<string>(reader);
return (Inner)str; // Or however you want to convert string to Inner
} else {
return default(Inner);
}
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
throw new System.NotImplementedException();
}
}