We Keep Coding

Create custom JSON converter using Newtonsoft.JSON library to get additional data without overriding the CanRead and CanWrite flags

I am currently working on a Custom JSON converter to be used in a WebAPI project. The requirement is - I have a DTO object having some properties. The APIs can be consumed by multiple clients. Depending upon a client few of my DTO Entities might have some additional data apart from the properties already present in the DTO Model. I need to create a custom JSON converter to Serialize and Deserialize this data.
//DTO
class AbcDTO
{
public string Prop1 { get; set; }
public string Prop2 { get; set; }
public List<AdditionalProperty> AdditionalData { get; set; }
}
//AdditionalProperty class
class AdditionalProperty
{
public string Name { get; set; }
public object Value { get; set; }
}
//Request JSON Body
{
"Prop1": "Val1",
"Prop2": "Val2",
"AdditionalProp3": "Val3",
"AdditionalProp4": "Val4"
}
//After Deserialization the object should be as below
AbcDTO dto = {
Prop1 = "Val1",
Prop2 = "Val2",
AdditionalData = [
{ Name = "AdditionalProp3", Value = "Val3" },
{ Name = "AdditionalProp4", Value = "Val4" }]
}
//After Serialization of the above dto object the JSON should convert back to the Request JSON Body format
We don't want to use the JsonExtensionData attribute provided by Newtonsoft.JSON as we would need to keep the property as Dictionary<string, JToken> -- but we don't want to pass JToken to below layers.
Created a custom JSON converter -
class CustomJsonConverter : JsonConverter
{
bool _canWrite = true;
bool _canRead = true;
public override bool CanConvert(Type objectType)
{
return typeof(IEntity).IsAssignableFrom(objectType);
}
public override bool CanWrite
{
get
{
return _canWrite;
}
}
public override bool CanRead
{
get
{
return _canRead;
}
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JObject jObject = JObject.Load(reader);
PropertyInfo[] availablePropertyNames = objectType.GetProperties();
List<AdditionalProperties> additionalData = new List<AdditionalProperties>();
IEntity obj;
_canRead = false;
obj = (IEntity)jObject.ToObject(objectType);
_canRead = true;
IEnumerable<JProperty> properties = jObject.Properties();
foreach (JProperty prop in properties)
{
if (availablePropertyNames.Count(x => x.Name.Equals(prop.Name)) == 0)
{
AdditionalProperties addProp = new AdditionalProperties
{
Name = prop.Name,
Value = prop.Value.ToObject<object>(),
};
additionalData.Add(addProp);
}
}
obj.AdditionalData = additionalData;
return obj;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
IEntity obj = (IEntity)value;
List<AdditionalProperties> additionalData = obj.AdditionalData;
JObject jObj;
_canWrite = false;
jObj = (JObject)JToken.FromObject(obj);
_canWrite = true;
jObj.Remove("AdditionalData");
foreach (AdditionalProperties data in additionalData)
{
jObj.Add(data.Name, JToken.FromObject(data.Value));
}
jObj.WriteTo(writer);
}
}
WebAPI ContractResolver creates 1 JSON converter per Entity. Now the issue is _canRead and _canWrite are not thread-safe. Need to use them to use the base implementation provided by Newtonsoft. If we don't use them, the ToObject and FromObject method again calls the custom converter methods internally resulting in infinite recursion. Using them with logs, reduces performance. Is there any way we can create a custom converter using the base implementation of Newtonsoft.JSON serialization/deserialization without using canRead and canWrite flags?
I can also have reference type child properties - say Person contains Address. I want to capture additional data for both Parent and Child entities. The additional data will not contain data of reference type.

It's possible to disable the converter using a thread static variable or ThreadLocal<T> member, as shown in JSON.Net throws StackOverflowException when using JsonConvert or Generic method of modifying JSON before being returned to client. However, I'd like to suggest a simpler way of solving your problem.
You wrote, We dont want to use the JsonExtensionData attribute provided by Newtonsoft.JSON as we need to keep the property as Dictionary and we dont want to pass JToken to below layers. It is not necessary for the extension data dictionary to have values of type JToken. Values of type object are supported for extension data dictionaries, e.g.:
class AbcDTO
{
public AbcDTO() { this.AdditionalData = new Dictionary<string, object>(); }
public string Prop1 { get; set; }
public string Prop2 { get; set; }
[JsonExtensionData]
public Dictionary<string, object> AdditionalData { get; private set; }
}
When the extension data dictionary is of type Dictionary<string, object>, Json.NET will deserialize JSON primitive values to their equivalent .Net primitives -- string, bool, long and so on -- rather than to JValue objects. Only when encountering an additional property whose value is a JSON object or array will a JToken be added to the dictionary, in which case you can use the answers from How do I use JSON.NET to deserialize into nested/recursive Dictionary and List? to convert the JToken to a conventional .Net type. (However, your question states that The additional data will not contain data of reference type, so this should not be necessary.)
Using [JsonExtensionData] in this manner completely avoids the need for a converter while also deserializing primitives as per your requirements, and thus seems much simpler than the original design shown in the question.
Sample .Net fiddle demonstrating that extension properties can be deserialized into AbcDTO and asserting that none of them are of type JToken.

Declared properties won't be serialized if its declaring type is a dynamic object

Let's say I've the following dynamic object:
public class SomeDynamicObject : DynamicObject
{
public string Text { get; set; }
}
If I serialize it using JsonConvert.SerializeObject(new SomeDynamicObject { Text = "hello world" }) it'll return {} instead of { "Text": "hello world" }.
I suspect the issue is that JSON.NET thinks it's a full dynamic object while my case is a dynamic object with declared members.
Is there any serialization settings or built-in converter that could be configured so JSON.NET can serialize both kinds of members?
To avoid confusion
Actual use case: I don't know which will be the types being serialized but I need to cover the whole use case of serializing declared properties of a dynamic object.
That is, I can't use attributes. That's why I'm asking if there's some converter or a serialization setting that can generalize this use case.

Update for non-attribute converter
Since you can't decorate, you lose a lot of power. Once the JsonWriter has converted to a JObject, the dynamic properties appear to be lost.
However, you can always use a little reflection in a custom converter's WriteJson method to serialize non-dynamic types.
public class SomeDynamicObject : DynamicObject
{
public string Text { get; set; }
public DynamicObject DynamicProperty { get; set; }
}
public class CustomDynamicConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return true;
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
JObject jObject = JObject.Load(reader);
var target = Activator.CreateInstance(objectType);
//Create a new reader for this jObject, and set all properties to match the original reader.
JsonReader jObjectReader = jObject.CreateReader();
jObjectReader.Culture = reader.Culture;
jObjectReader.DateParseHandling = reader.DateParseHandling;
jObjectReader.DateTimeZoneHandling = reader.DateTimeZoneHandling;
jObjectReader.FloatParseHandling = reader.FloatParseHandling;
// Populate the object properties
serializer.Populate(jObjectReader, target);
return target;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var properties = value.GetType().GetProperties().Where(x => x.PropertyType != typeof(DynamicObject)).ToList();
JObject o = (JObject)JToken.FromObject(value);
properties.ForEach(x =>
{
o.AddFirst(new JProperty(x.Name, x.GetValue(value)));
});
o.WriteTo(writer);
}
}
If you explicitly decorate your properties with [JsonProperty], the serializer will pick them up, even if the containing type is dynamic.
public class SomeDynamicObject : DynamicObject
{
[JsonProperty]
public string Text { get; set; }
}
when serialized correctly outputs:
{"Text":"hello world"}

How to serialize all nulls to empty strings using Json.Net

I would like to serialize all nulls from string and nullable types to empty strings. I looked into using a custom JsonConverter but properties with nulls are not passed into the converter. I looked into using a contract resolver and value provider but it won't let me set the value of an int? to an empty string.
What seems most promising is using a custom JsonWriter that overrides the WriteNull method, but when I instantiate that in the converter's WriteJson method, it doesn't write anything out.
public class NullJsonWriter : JsonTextWriter
{
public NullJsonWriter(TextWriter writer) : base(writer)
{
}
public override void WriteNull()
{
base.WriteValue(string.Empty);
}
}
public class GamesJsonConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(IEnumerable<IGame>).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
writer = new NullJsonWriter(new StringWriter(new StringBuilder()));
writer.WriteStartObject();
if (typeof (IEnumerable).IsAssignableFrom(value.GetType()))
{
var list = (IEnumerable<IGame>)value;
if (!list.Any())
{
writer.WriteEndObject();
return;
}
var properties = list.First().GetType().GetProperties();
PropertyInfo gameId = null;
foreach (var prop in properties)
{
if (prop.Name == "GameId")
{
gameId = prop;
break;
}
}
for (int i = 0; i < list.Count(); i++)
{
writer.WritePropertyName(string.Format("{0}", gameId.GetValue(list.ElementAt(i))));
serializer.Serialize(writer, list.ElementAt(i));
}
}
writer.WriteEndObject();
}
I can see JSON text in the StringBuilder of the writer but it doesn't actually get written out to my webpage.

I'm not sure I understand the reasoning of why you want to write out an empty string in place of all nulls, even those that might represent a Nullable<int>, or other object. The problem with that strategy is it makes the JSON more difficult to deal with during deserialization: if the consumer is expecting a Nullable<int> and they get a string instead, it leads to errors, confusion, and essentially forces the consuming code to use a special converter or weakly typed objects (e.g. JObject, dynamic) to get around the mismatch. But let's put that aside for now and say that you have your reasons.
You can definitely do what you want by subclassing the JsonTextWriter class and overriding the WriteNull method to write an empty string instead. The trick is that you must instantiate your own JsonSerializer instead of using JsonConvert.SerializeObject, since the latter does not have any overloads that accept a JsonWriter. Here is a short proof-of-concept that shows that this idea will work:
class Program
{
static void Main(string[] args)
{
// Set up a dummy object with some data. The object also has a bunch
// of properties that are never set, so they have null values by default.
Foo foo = new Foo
{
String = "test",
Int = 123,
Decimal = 3.14M,
Object = new Bar { Name = "obj1" },
Array = new Bar[]
{
new Bar { Name = "obj2" }
}
};
// The serializer writes to the custom NullJsonWriter, which
// writes to the StringWriter, which writes to the StringBuilder.
// We could also use a StreamWriter in place of the StringWriter
// if we wanted to write to a file or web response or some other stream.
StringBuilder sb = new StringBuilder();
using (StringWriter sw = new StringWriter(sb))
using (NullJsonWriter njw = new NullJsonWriter(sw))
{
JsonSerializer ser = new JsonSerializer();
ser.Formatting = Formatting.Indented;
ser.Serialize(njw, foo);
}
// Get the JSON result from the StringBuilder and write it to the console.
Console.WriteLine(sb.ToString());
}
}
class Foo
{
public string String { get; set; }
public string NullString { get; set; }
public int? Int { get; set; }
public int? NullInt { get; set; }
public decimal? Decimal { get; set; }
public decimal? NullDecimal { get; set; }
public Bar Object { get; set; }
public Bar NullObject { get; set; }
public Bar[] Array { get; set; }
public Bar[] NullArray { get; set; }
}
class Bar
{
public string Name { get; set; }
}
public class NullJsonWriter : JsonTextWriter
{
public NullJsonWriter(TextWriter writer) : base(writer)
{
}
public override void WriteNull()
{
base.WriteValue(string.Empty);
}
}
Here is the output:
{
"String": "test",
"NullString": "",
"Int": 123,
"NullInt": "",
"Decimal": 3.14,
"NullDecimal": "",
"Object": {
"Name": "obj1"
},
"NullObject": "",
"Array": [
{
"Name": "obj2"
}
],
"NullArray": ""
}
Fiddle: https://dotnetfiddle.net/QpfG8D
Now let's talk about why this approach did not work as expected inside your converter. When JSON.Net calls into your converter, it already has a JsonWriter instantiated, which gets passed into the writer parameter of the WriteJson method. Instead of writing to that instance, what you have done is overwritten this variable with a new NullJsonWriter instance. But remember, replacing the contents of a reference variable does not replace the original instance that variable refers to. You are successfully writing to your new writer instance in the converter, but all the output that you are writing is going into the StringBuilder which you instantiated at the beginning of the WriteJson method. Since you never take the JSON result from the StringBuilder and do something with it, it is simply thrown away at the end of the method. Meanwhile, Json.Net continues to use its original JsonWriter instance, to which you have not written anything inside your converter. So, that is why you are not seeing your customized JSON in the final output.
There are a couple of ways to fix your code. One approach is just to use a new variable when instantiating the NullJsonWriter in your converter instead of hijacking the writer variable. Then, at the end of the WriteJson method, get the JSON from the StringBuilder and write it to the original writer using WriteRawValue method on the original writer.
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
StringBuilder sb = new StringBuilder();
using (var innerWriter = new NullJsonWriter(new StringWriter(sb))
{
innerWriter.WriteStartObject();
// ... (snip) ...
innerWriter.WriteEndObject();
}
writer.WriteRawValue(sb.ToString());
}
Another approach, depending on your intended result, is not to use a separate writer inside your converter at all, but instead to create the NullJsonWriter instance at the very beginning of serialization and pass it to the outer serializer (as I did in the proof-of-concept earlier in the post). When Json.Net calls into your converter, the writer that is passed to WriteJson will be your custom NullJsonWriter, so at that point you can write to it naturally without having to jump through hoops. Keep in mind, however, that with this approach your entire JSON will have the nulls replaced with empty strings, not just the IEnumerable<IGame> that your converter handles. If you are trying to confine the special behavior to just your converter, then this approach will not work for you. It really depends on what you are trying to accomplish-- do you want the nulls replaced everywhere or just in part of the JSON? I did not get a good sense of this from your question.
Anyway, hope this helps.

JsonSerializer _jsonWriter = new JsonSerializer
{
NullValueHandling = NullValueHandling.Include
};
Use this to make the serializer not ignore null values, then you should be able to override the serialization with the code you already have :)

Json.net deserialization null guid case

I'm deserializing an object using Json.NET that contains a private field of type Guid and a public property for that field. When the value for my Guid is null in my json I want to assign Guid.Empty to my field.
public class MyClass
{
private Guid property;
public Guid Property
{
get { return property; }
set
{
if (value == null)
{
property = Guid.Empty;
}
else
{
property = value;
}
}
}
}
But the deserializer wants to access the private field, cause I get this error when I try to deserialize:
Error converting value {null} to type 'System.Guid'. Path
'[0].property', line 6, position 26.
How can I make it ignore the private field and use the public property instead?

Json.NET refuses to set a null value for a Guid because it is a non-nullable value type. Try typing (Guid)null in the Immediate Window and you will see an error message indicating that this conversion cannot be made in .Net.
To work around this, you have a couple of options:
Create a Guid? nullable proxy property. It can be private if you desire as long as it has a [JsonProperty] attribute:
public class MyClass
{
[JsonIgnore]
public Guid Property { get; set; }
[JsonProperty("Property")]
Guid? NullableProperty { get { return Property == Guid.Empty ? null : (Guid?)Property; } set { Property = (value == null ? Guid.Empty : value.Value); } }
}
Create a JsonConverter that converts a null Json token to a default Guid value:
public class NullToDefaultConverter<T> : JsonConverter where T : struct
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(T);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var token = JToken.Load(reader);
if (token == null || token.Type == JTokenType.Null)
return default(T);
return token.ToObject(objectType); // Deserialize using default serializer
}
// Return false instead if you don't want default values to be written as null
public override bool CanWrite { get { return true; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
if (EqualityComparer<T>.Default.Equals((T)value, default(T)))
writer.WriteNull();
else
writer.WriteValue(value);
}
}
Then apply it to your type as follows:
public class MyClass
{
[JsonConverter(typeof(NullToDefaultConverter<Guid>))]
public Guid Property { get; set; }
}
Alternatively, you can apply the converter to all values of type T by adding the converter to JsonSerializerSettings.Converters. And, to register such a converter globally, see e.g.How to set custom JsonSerializerSettings for Json.NET in MVC 4 Web API? for Web API, Setting JsonConvert.DefaultSettings asp net core 2.0 not working as expected for ASP.NET Core or Registering a custom JsonConverter globally in Json.Net for a console app.
If you do register the converter globally for a console app, you may need to disable it for recursive calls as shown in JSON.Net throws StackOverflowException when using [JsonConvert()].
If you only need to deserialize a null value for a Guid and not re-serialize it as such, you can apply [JsonProperty(NullValueHandling = NullValueHandling.Ignore)] to the Guid property, and null values will ignored despite being invalid Guid values:
public class MyClass
{
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
public Guid Property { get; set; }
}
Of course if you do this your Guid will be re-serialized as "00000000-0000-0000-0000-000000000000". To ameliorate that you could apply DefaultValueHandling = DefaultValueHandling.Ignore which will cause empty Guid values to be omitted during serialization:
[JsonProperty(NullValueHandling = NullValueHandling.Ignore, DefaultValueHandling = DefaultValueHandling.Ignore)]
public Guid Property { get; set; }
Note that if a parameterized constructor called during deserialization has a non-nullable Guid argument with the same name, a different approach may be required.

JsonConvert attributes appear to be getting skipped over. Why?

I have a number of properties that I'm trying to use Json.NET to serialize. Complex objects work fine as all I have to put above them are the [JsonProperty("name")] attributes.
Unfortunately, I have been struggling to get this to work with any enums. Ideally, I'd like to be able to define the property by just saying something like:
MyValue = ThisEnum.First;
and have the property within my class automatically serialize the enum to the appropriate changed value name (as this question asked: Control enum value format during (de)serialization via attributes. Unfortunately, I have written a number of custom converters and attempted to apply the suggestions of the people in one of my previous questions (Can't get enum to convert to json properly using Json.NET) and despite the respondents in that thread giving answers that look like they'd work, in my solution, they don't have any effect whatsoever.
For example, given this class of properties:
public class Animal
{
[JsonProperty("id")]
public string Id {get;set;}
[JsonProperty("types")]
[JsonConverter(typeof(StringEnumConverter))]
public AnimalType Types {get;set;}
[JsonProperty("created")]
[JsonConverter(typeof(MyDateTimeToSecondsSinceEpochConverter))]
public DateTime Created {get;set;}
}
Within the AnimalType enum:
public enum AnimalType
{
[EnumMember(Value="black_rhino")]
BlackRhino,
[EnumMember(Value="wild_pig")]
WildPig,
[EnumMember(Value="chicken")]
Chicken
}
Within the MyDateTimeToSecondsSinceEpochConverter class:
public class MyDateTimeToSecondsSinceEpochConverter : DateTimeConverterBase
{
public override WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
writer.WriteRawValue(#"""\/Date(" + ConvertDateTimeToEpoch((DateTime)value).ToString() + #")\/""");
}
public override ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.Value == null) return null;
if (reader.TokenType == JsonToken.Integer)
return ConvertEpochToDateTime((long)reader.Value);
return DateTime.Parse(reader.Value.ToString());
}
public DateTime ConvertEpochToDateTime(long seconds)
{
return new DateTime(1970, 1, 1).AddSeconds(seconds);
}
public long ConvertDateTimeToEpoch(DateTime dateTime)
{
var epochStart = new DateTime(1970,1,1);
if (dateTime < epochStart) return 0;
return Convert.ToInt64(dateTime.Subtract(epochStart).TotalSeconds);
}
}
In my calling method, I can have something like:
var data = new Animal {
Id = "123abc",
Types = AnimalType.BlackRhino,
Created = new DateTime(2014,3,15)
}
I use a method through which I step through each of the properties of the object with the simple intention of listing each of them and the HTML encoded output of the value and putting them all in a string (that I'll later attach to a URL):
public static string GetString(Animal animal)
{
var result = "";
foreach( var property in animal.GetType().GetProperties(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance)
{
foreach(var attribute in property.GetCustomAttributes(false))
{
var name = (JsonPropertyAttribute)attribute;
var value = property.GetValue(animal, null);
if (value != null)
{
if (!string.Empty(result))
result += "&";
result += string.Format("{0}={1}", name.PropertyName, HttpUtility.UrlEncode(value.ToString()));
}
}
}
return result;
}
Unfortunately, when I run this, I see:
'id=abc123&type=BlackRhino&created=3%2f15%2f2014+12%3a00%3a00+AM'
Does anyone have any ideas as to how I can get the two JsonConverters to actually do some converting? The JsonProperty name is obviously changing, but 'BlackRhino' didn't change to 'black_rhino' nor did the DateTime convert to a long.
I'm ideally looking for something in attribute land or using these converters so that I don't have to write a get/set for every single enum I use in the project down the road, but can rather just apply the attribute and be done with it.
Thanks!

You can have Json.NET turn your object into a string and then into a dictionary, which is a collection of name/value pairs like a query string is.
public static string GetString(Animal animal)
{
var result = "";
var serialized = JsonConvert.SerializeObject(animal);
var dict = JsonConvert.DeserializeObject<IDictionary<string, string>>(serialized);
foreach (var pair in dict)
{
if (!string.IsNullOrEmpty(result))
result += "&";
result += string.Format("{0}={1}", pair.Key, HttpUtility.UrlEncode(pair.Value.ToString()));
}
return result;
}
With your example, this is:
id=123abc&types=black_rhino&created=%2fDate(1394841600)%2f