We Keep Coding

Json.Net - deserialize json to a dictionary with special handling for a specific field

https://dotnetfiddle.net/ka6XVw - Fiddle with example type structure
Suppose I have a class that implements IDictionary<string, T>. Json.Net can deserialize such types out of the box, creating an instance of the type and using its indexer to populate the dictionary. The issue is that this class also inherits a string Error property marked with JsonProperty attribute from its base class, and I'd like this property to be populated whenever the input json contains an error field. However, when deserializing an IDictionary Json.Net considers all fields to be dictionary entries and tries to add the value with the error key to the dictionary.
What is the simplest and cleanest way to deserialize the json into a dictionary and the error field into the Error property? Please note that the class is generic, so JsonExtensionData is not an option (without casting its values to the provided type).
Sample valid dictionary json: { 'foo': '1', 'bar': '2' }
Sample error json { 'error': 'blah' }

I've derived a converter solution from this question. Basically, you attach a converter to your DictionaryResponse class, and interpret the incoming JSON yourself. I was lazy enough to use a JObject for parsing:
class DictionaryResponseConverter : JsonConverter<ResponseBase>
{
public override ResponseBase ReadJson(
JsonReader reader, Type objectType,
ResponseBase existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
// find the correct T and call the internal function through reflection
// as DictionaryResponse<T> is sealed, we don't care about inheritance
return (ResponseBase)GetType()
.GetMethod(nameof(InternalReadJson),
BindingFlags.Instance | BindingFlags.NonPublic)
.MakeGenericMethod(objectType.GetGenericArguments()[0])
.Invoke(this, new object[]
{
reader,
existingValue,
hasExistingValue,
serializer
});
}
DictionaryResponse<T> InternalReadJson<T>(
JsonReader reader,
DictionaryResponse<T> existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
var obj = JObject.Load(reader);
var error = (string)obj["error"];
bool hadError = obj.Remove("error");
//var result = new DictionaryResponse<T>();
var result = hasExistingValue ? existingValue : new DictionaryResponse<T>();
foreach (var kvp in obj)
result[kvp.Key] = kvp.Value.ToObject<T>();
if (hadError)
result.Error = error;
return result;
}
public override void WriteJson(
JsonWriter writer, ResponseBase value, JsonSerializer serializer)
{
// don't care about serialization
throw new NotImplementedException();
}
}
[JsonConverter(typeof(DictionaryResponseConverter))]
internal sealed class DictionaryResponse<T> : ResponseBase, IDictionary<string, T>
{
...

How can I serialise PSObjects in C# with JSON.NET?

I am writing a Cmdlet and need to pass object structures into an API client that may contain PSObjects. Currently, these serialise as a JSON string containing CLIXML. Instead, I need it to be treated like an object (including the NoteProperties in PSObject.Properties as properties, and recursively serialising their values).
I tried writing my own JsonConverter but for some reason it only gets called for the top level object, not for nested PSObjects:
public class PSObjectJsonConverter : JsonConverter {
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
if (value is PSObject) {
JObject obj = new JObject();
foreach (var prop in ((PSObject)value).Properties) {
obj.Add(new JProperty(prop.Name, value));
}
obj.WriteTo(writer);
} else {
JToken token = JToken.FromObject(value);
token.WriteTo(writer);
}
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
throw new NotImplementedException();
}
public override bool CanRead {
get { return false; }
}
public override bool CanConvert(Type objectType) {
return true;
}
}
Additionally, I am using serializing to camel case using CamelCasePropertyNamesContractResolver. Is there a way to make the converter respect that?

The following converter should correctly serialize recursively nested objects of type PSObject:
public class PSObjectJsonConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(PSObject).IsAssignableFrom(objectType);
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var psObj = (PSObject)value;
writer.WriteStartObject();
foreach (var prop in psObj.Properties)
{
//Probably we shouldn't try to serialize a property that can't be read.
//https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.pspropertyinfo.isgettable?view=powershellsdk-1.1.0#System_Management_Automation_PSPropertyInfo_IsGettable
if (!prop.IsGettable)
continue;
writer.WritePropertyName(prop.Name);
serializer.Serialize(writer, prop.Value);
}
writer.WriteEndObject();
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override bool CanRead { get { return false; } }
}
Notes:
In WriteJson you serialize the incoming object value as the value of each property. Surely you meant prop.Value.
By only returning true from CanConvert() when the incoming object type is of type PSObject, you avoid the need to implement default serialization for non-PSObject types in WriteJson().
When you call JToken.FromObject(value) you are not using the incoming JsonSerializer serializer. Thus, any JsonSerializerSettings (including converters) will be lost. In theory you could use JToken.FromObject(Object, JsonSerializer) instead, which would preserve settings, but if you did, you would encounter the bug described in JSON.Net throws StackOverflowException when using [JsonConvert()]. Luckily, since we now return false from CanConvert when default serialization is required, this is no longer necessary.
There is no need to construct an intermediate JObject. You can write directly to the JsonWriter, which will be somewhat more performant.
Update: Additionally, I am using serializing to camel case using CamelCasePropertyNamesContractResolver. Is there a way to make the converter respect that?
Once you introduce a custom JsonConverter for your type, you need to do everything manually, including remapping of property names. Here's a version of WriteJson() that handles this by using DefaultContractResolver.NamingStrategy:
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var psObj = (PSObject)value;
writer.WriteStartObject();
var resolver = serializer.ContractResolver as DefaultContractResolver;
var strategy = (resolver == null ? null : resolver.NamingStrategy) ?? new DefaultNamingStrategy();
foreach (var prop in psObj.Properties)
{
//Probably we shouldn't try to serialize a property that can't be read.
//https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.pspropertyinfo.isgettable?view=powershellsdk-1.1.0#System_Management_Automation_PSPropertyInfo_IsGettable
if (!prop.IsGettable)
continue;
writer.WritePropertyName(strategy.GetPropertyName(prop.Name, false));
serializer.Serialize(writer, prop.Value);
}
writer.WriteEndObject();
}
Note that naming strategies were introduced in Json.NET 9.0.1 so if you are using an earlier version you will need to create your own camel case name mapper such as the one shown in this answer.

JSON.net serializes a json array into a JArray when the destination is an object. How can I change that?

I have a single level json that I want to deserialize into a Dictionary<string,object> using Json.Net.
The dictionary's value can be a primitive, string or a (primitive\string) array.
The deserialization knows how to handle primitives and strings, but when it gets to an array of primitives it deserializes it into a JArray (instead of a primitive array).
Here's a small code example of what I mean:
string jsonStr = #"{""obj"": 7, ""arr"": ['1','2','3']}";
Dictionary<string, object> dict = JsonConvert.DeserializeObject<Dictionary<string, object>>(jsonStr);
dict["obj"].GetType(); // long
dict["arr"].GetType(); // JArray. I would like this to be string[].
I'm looking for a way I can interfere in the deserialization process and create a primitive array instead of getting stuck with a JArray.
I've tried doing it with the JsonSerializerSettings, but couldn't nail the spot.

Here is a hacky way of doing this. I create a custom JsonConverter class which accepts 2 generic arguments: T1 and T2. T1 gives the type of the array (in this case string) and T2 gives the type of the other object (in this case long). I assume that we basically want a Dictionary, but that part could definitely be improved.
using System;
using System.Collections.Generic;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string jsonStr = #"{""obj"": 7, ""arr"": ['1','2','3']}";
Dictionary<string, object> dict = JsonConvert.DeserializeObject<Dictionary<string, object>>(jsonStr, new SpecialConverter<string, long>());
dict["obj"].GetType(); // long
dict["arr"].GetType(); // string[].
}
class SpecialConverter<T1, T2> : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return true;
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JToken token = JToken.Load(reader);
var result = new Dictionary<string, object>();
if (result.GetType() == objectType)
{
foreach (var item in token)
{
var prop = (JProperty)item;
if (prop.Value.Type == JTokenType.Array)
{
result.Add(prop.Name, prop.Value.ToObject<T1[]>());
}
else
{
result.Add(prop.Name, prop.Value.ToObject<T2>());
}
}
return result;
}
else
{
return null;
}
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
}
}

The reason it does not convert into an array is because you are telling JsonConvert to convert it to an Dictionary<string, object> and that is exactly what its doing based on the fact that it hasn't got a specific Object to convert to , hence its has "guess" picked JArray, If you can have multiple types for the dictionary's value, best thing would be to convert it to dynamic object. This will give you an array if the value of "arr" is array or a string if the value is a simple string.
string jsonStr = #"{""obj"": 7, ""arr"": ['1','2','3']}";
dynamic dict = JsonConvert.DeserializeObject(jsonStr);
var arr = dict.arr;
string firstelement = arr[0]; // print "1"

JSON.Net serializing Enums to strings in dictionaries by default - how to make it serialize to int?

Why does my serialized JSON end up as
{"Gender":1,"Dictionary":{"Male":100,"Female":200}}
i.e. why do the enums serialize to their value, but when they form they key to the dictionary they are converted to their key?
How do I make them be ints in the dictionary, and why isn't this the default behaviour?
I'd expect the following output
{"Gender":1,"Dictionary":{"0":100,"1":200}}
My code:
public void foo()
{
var testClass = new TestClass();
testClass.Gender = Gender.Female;
testClass.Dictionary.Add(Gender.Male, 100);
testClass.Dictionary.Add(Gender.Female, 200);
var serializeObject = JsonConvert.SerializeObject(testClass);
// serializeObject == {"Gender":1,"Dictionary":{"Male":100,"Female":200}}
}
public enum Gender
{
Male = 0,
Female = 1
}
public class TestClass
{
public Gender Gender { get; set; }
public IDictionary<Gender, int> Dictionary { get; set; }
public TestClass()
{
this.Dictionary = new Dictionary<Gender, int>();
}
}
}

The reason why Gender enum is serialized to its value when used as property value, but it is serialized to its string representation when used as dictionary key is the following:
When used as property value JSON.NET serializer first writes the property name and after that the property value. For the example you posted, JSON.NET will write "Gender" as property name (notice that it writes a string), than will try to resolve the value of the property. The value of the property is of type enum which JSON.NET handles as Int32 and it writes the number representation of the enum
When serializing the dictionary, the keys are written as property names, so the JSON.NET serializer writes the string representation of the enum. If you switch the types of the keys and values in the dictionary (Dictionary<int, Gender> instead of Dictionary<Gender, int>, you'll verify that the enum will be serialized with its Int32 representation.
To achieve what you want with the example you posted, you'll need to write custom JsonConverter for the Dictionary property. Something like this:
public class DictionaryConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var dictionary = (Dictionary<Gender, int>) value;
writer.WriteStartObject();
foreach (KeyValuePair<Gender, int> pair in dictionary)
{
writer.WritePropertyName(((int)pair.Key).ToString());
writer.WriteValue(pair.Value);
}
writer.WriteEndObject();
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var jObject = JObject.Load(reader);
var maleValue = int.Parse(jObject[((int) Gender.Male).ToString()].ToString());
var femaleValue = int.Parse(jObject[((int)Gender.Female).ToString()].ToString());
(existingValue as Dictionary<Gender, int>).Add(Gender.Male, maleValue);
(existingValue as Dictionary<Gender, int>).Add(Gender.Female, femaleValue);
return existingValue;
}
public override bool CanConvert(Type objectType)
{
return typeof (IDictionary<Gender, int>) == objectType;
}
}
and decorate the property in the TestClass:
public class TestClass
{
public Gender Gender { get; set; }
[JsonConverter(typeof(DictionaryConverter))]
public IDictionary<Gender, int> Dictionary { get; set; }
public TestClass()
{
this.Dictionary = new Dictionary<Gender, int>();
}
}
When calling the following line for serialization:
var serializeObject = JsonConvert.SerializeObject(testClass);
you'll get the desired output:
{"Gender":1,"Dictionary":{"0":100,"1":200}}

I often find myself facing this issue so I did a JsonConverter that can handle any kind of dictionnary with an Enum type as key:
public class DictionaryWithEnumKeyConverter<T, U> : JsonConverter where T : System.Enum
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var dictionary = (Dictionary<T, U>)value;
writer.WriteStartObject();
foreach (KeyValuePair<T, U> pair in dictionary)
{
writer.WritePropertyName(Convert.ToInt32(pair.Key).ToString());
serializer.Serialize(writer, pair.Value);
}
writer.WriteEndObject();
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var result = new Dictionary<T, U>();
var jObject = JObject.Load(reader);
foreach (var x in jObject)
{
T key = (T) (object) int.Parse(x.Key); // A bit of boxing here but hey
U value = (U) x.Value.ToObject(typeof(U));
result.Add(key, value);
}
return result;
}
public override bool CanConvert(Type objectType)
{
return typeof(IDictionary<T, U>) == objectType;
}
}
NB: This will not handle Dictionnary<Enum, Dictionnary<Enum, T>

The answer from Ilija Dimov covers why it happens, but the suggested converter only works for this specific case.
Here's a reusable converter which formats enum keys as their value, for any enum key in any Dictionary<,>/IDictionary<,> field:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using Newtonsoft.Json;
/// <summary>A Json.NET converter which formats enum dictionary keys as their underlying value instead of their name.</summary>
public class DictionaryNumericEnumKeysConverter : JsonConverter
{
public override bool CanRead => false; // the default converter handles numeric keys fine
public override bool CanWrite => true;
public override bool CanConvert(Type objectType)
{
return this.TryGetEnumType(objectType, out _);
}
public override object ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
{
throw new NotSupportedException($"Reading isn't implemented by the {nameof(DictionaryNumericEnumKeysConverter)} converter."); // shouldn't be called since we set CanRead to false
}
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
{
// handle null
if (value is null)
{
writer.WriteNull();
return;
}
// get dictionary & key type
if (value is not IDictionary dictionary || !this.TryGetEnumType(value.GetType(), out Type? enumType))
throw new InvalidOperationException($"Can't parse value type '{value.GetType().FullName}' as a supported dictionary type."); // shouldn't be possible since we check in CanConvert
Type enumValueType = Enum.GetUnderlyingType(enumType);
// serialize
writer.WriteStartObject();
foreach (DictionaryEntry pair in dictionary)
{
writer.WritePropertyName(Convert.ChangeType(pair.Key, enumValueType).ToString()!);
serializer.Serialize(writer, pair.Value);
}
writer.WriteEndObject();
}
/// <summary>Get the enum type for a dictionary's keys, if applicable.</summary>
/// <param name="objectType">The possible dictionary type.</param>
/// <param name="keyType">The dictionary key type.</param>
/// <returns>Returns whether the <paramref name="objectType"/> is a supported dictionary and the <paramref name="keyType"/> was extracted.</returns>
private bool TryGetEnumType(Type objectType, [NotNullWhen(true)] out Type? keyType)
{
// ignore if type can't be dictionary
if (!objectType.IsGenericType || objectType.IsValueType)
{
keyType = null;
return false;
}
// ignore if not a supported dictionary
{
Type genericType = objectType.GetGenericTypeDefinition();
if (genericType != typeof(IDictionary<,>) && genericType != typeof(Dictionary<,>))
{
keyType = null;
return false;
}
}
// extract key type
keyType = objectType.GetGenericArguments().First();
if (!keyType.IsEnum)
keyType = null;
return keyType != null;
}
}
You can enable it on a specific field:
[JsonConverter(typeof(DictionaryNumericEnumKeysConverter))]
public IDictionary<Gender, int> Dictionary { get; set; }
Or enable it for all dictionaries with enum keys:
JsonConvert.DefaultSettings = () => new JsonSerializerSettings
{
Converters = new List<JsonConverter>
{
new DictionaryNumericEnumKeysConverter()
}
};

If you're asking why this is the default behavior as a matter of compu-prudence, there are at least a couple of really good reasons IMO, but both ultimately have to do with interoperability, i.e., the likelihood that your JSON data is getting exchanged with another system outside your own and/or that uses another language and/or doesn't share your C# code.
First, enum number values in C# can change in different assembly versions if you insert more and don't specify numbers yourself. Granted you specify the numbers here, but many people do not. Plus, even if you do specify the numbers, you're now committing to these values as part of your API contract for all time, or else make a breaking change. And even if the only consumer of this JSON data is your own code, unless you use some kind of automatic Typescript generator based on your C# (which I do and should be done more often!!) you have at least two places you'd need to update if you want to change these numbers. So by serializing using the enum name as the key rather than the number value, consumers can remain oblivious to the numeric values and have no possibility of breaking when the enum numbers change.
Second, the string values are just more user friendly to consumers. Anyone looking at the raw JSON data would likely be able to get a good understanding of the contents of the dictionary without the need for any documentation. Better still, the consumer doesn't have to be keeping independent track of the number values for each key in their own code, which would just be another opportunity for error on their end.
So as others noted, you can change this behavior if you want, but in terms of picking a default behavior that captures the practice of the most people and the original intent behind JSON, this seems like the right approach.

Json.NET: Deserializing nested dictionaries

When deserializing an object to a Dictionary (JsonConvert.DeserializeObject<IDictionary<string,object>>(json)) nested objects are deserialized to JObjects. Is it possible to force nested objects to be deserialized to Dictionarys?

I found a way to convert all nested objects to Dictionary<string,object> by providing a CustomCreationConverter implementation:
class MyConverter : CustomCreationConverter<IDictionary<string, object>>
{
public override IDictionary<string, object> Create(Type objectType)
{
return new Dictionary<string, object>();
}
public override bool CanConvert(Type objectType)
{
// in addition to handling IDictionary<string, object>
// we want to handle the deserialization of dict value
// which is of type object
return objectType == typeof(object) || base.CanConvert(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.StartObject
|| reader.TokenType == JsonToken.Null)
return base.ReadJson(reader, objectType, existingValue, serializer);
// if the next token is not an object
// then fall back on standard deserializer (strings, numbers etc.)
return serializer.Deserialize(reader);
}
}
class Program
{
static void Main(string[] args)
{
var json = File.ReadAllText(#"c:\test.json");
var obj = JsonConvert.DeserializeObject<IDictionary<string, object>>(
json, new JsonConverter[] {new MyConverter()});
}
}
Documentation: CustomCreationConverter with Json.NET

I had a very similar but slightly more complex need when I ran across this Q. At first I thought maybe I could adapt the accepted answer, but that seemed a bit complicated and I ended up taking a different approach. I was attempting to put a modern JSON layer on top of a legacy C++ API. I'll spare you the details of that, and just say the requirements boil down to:
JSON objects become Dictionary<string,object>.
JSON arrays become List<object>.
JSON values become the corresponding primitive CLR values.
The objects and arrays can be infinitely nested.
I first deserialize the request string into a Newtonsoft JSON object and then call my method to convert in accordance with the above requirements:
var jsonObject = JsonConvert.DeserializeObject(requestString);
var apiRequest = ToApiRequest(jsonObject);
// call the legacy C++ API ...
Here is my method that converts to the structure the API expects:
private static object ToApiRequest(object requestObject)
{
switch (requestObject)
{
case JObject jObject: // objects become Dictionary<string,object>
return ((IEnumerable<KeyValuePair<string, JToken>>) jObject).ToDictionary(j => j.Key, j => ToApiRequest(j.Value));
case JArray jArray: // arrays become List<object>
return jArray.Select(ToApiRequest).ToList();
case JValue jValue: // values just become the value
return jValue.Value;
default: // don't know what to do here
throw new Exception($"Unsupported type: {requestObject.GetType()}");
}
}
I hope that someone can find this approach useful.

Alternative/Update:
I needed to deserialize a dictionary of dictionaries of Strings and with current Json.NET (5.0) I did not had to create a CustomConverter, I just used (in VB.Net):
JsonConvert.DeserializeObject(Of IDictionary(Of String, IDictionary(Of String, String)))(jsonString)
Or, in C#:
JsonConvert.DeserializeObject<IDictionary<String, IDictionary<String, String>>(jsonString);

I have a nested/deep structure of "unknown" dictionaries that is serialized/deserialized to/from C# objects and JSON string. .NET 5.
If I use Newtonsoft it does not work automatically.
If I use System.Text.Json it works automatically.
//does NOT work (newtonDeserialized does not have the same data in the nested Dictionaries as in object):
var newtonSerialized = Newtonsoft.Json.JsonConvert.SerializeObject(object);
var newtonDeserialized = Newtonsoft.Json.JsonConvert.DeserializeObject<WaitlistResponse>(newtonSerialized);
//Works (netDeserialized have the same data in the nested Directories as in object):
var netSerialized = System.Text.Json.JsonSerializer.Serialize(object);
var netDeserialized = System.Text.Json.JsonSerializer.Deserialize<WaitlistResponse>(netSerialized);

#AlexD's accepted solution does not work ideally if there is an array in the json. It returns a JArray of JObject instead a List<Dictionary<string, object>>
This can be solved by modifying the ReadJson() method:
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.StartObject || reader.TokenType == JsonToken.Null)
return base.ReadJson(reader, objectType, existingValue, serializer);
//if it's an array serialize it as a list of dictionaries
if(reader.TokenType == JsonToken.ArrayStart)
return serializer.Deserialize(reader, typeof(List<Dictionary<string, object>>));
// if the next token is not an object
// then fall back on standard deserializer (strings, numbers etc.)
return serializer.Deserialize(reader);
}

In my case, not everything is nested dictionary. I also have an array that is key-value of primitive types and it throws exception when the object of the array is not a dictionary.
So, based on Phillip S' answer, I came up with
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue,
JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.StartObject || reader.TokenType == JsonToken.Null)
return base.ReadJson(reader, objectType, existingValue, serializer);
//if it's an array serialize it as a list of dictionaries
if (reader.TokenType == JsonToken.StartArray)
{
return serializer.Deserialize(reader, typeof(List<object>)); // instead of List<Dictionary<string, object>>
}
// if the next token is not an object
// then fall back on standard deserializer (strings, numbers etc.)
return serializer.Deserialize(reader);
}
Hope it helps those haven't got it working yet.

First of all you need to serialize your dictionary values.
var settings = new JsonSerializerSettings { TypeNameHandling= TypeNameHandling.All };
var serializeValues = JsonConvert.SerializeObject(nestedDictionaryValues, settings);
//After serialize using deserialize using the same TypeNameHandling.
var deserializeValues = JsonConvert.DeserializeObject(serializeValues, settings);