We Keep Coding

Controlling JSON .NET's reference ID generation

I'd like to be able to control how JSON .NET generates its meta reference IDs such as "$id": "1". Take the following code:
public class Person
{
public string Name { get; set; }
public Person Mother { get; set; }
}
.
var settings = new Newtonsoft.Json.JsonSerializerSettings
{
PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects,
Formatting = Newtonsoft.Json.Formatting.Indented
};
Newtonsoft.Json.JsonConvert.DefaultSettings = () => settings;
var person = new Person
{
Name = "bob",
Mother = new Person { Name = "jane" }
};
var personJson = JsonConvert.SerializeObject(person);
var motherJson = JsonConvert.SerializeObject(person.Mother);
The JSON for person looks like this:
{
"$id": "1",
"Name": "bob",
"Mother": {
"$id": "2",
"Name": "jane",
"Mother": null
}
}
However, if I serialize person.Mother directly, the JSON looks like this:
{
"$id": "1",
"Name": "jane",
"Mother": null
}
In the first JSON, Jane is "$id": "2", but serializing Jane directly is "$id": "1". This is the behavior I'd expect under normal conditions as the serializer assigns the IDs in the order it traverses the objects, but I'd really like to override the ID generation so that I could make it a hash of the object reference itself. This way, Jane would generate the same ID per running instance of the program every time regardless if serialized as a member of a parent or serialized individually.
UPDATE
Per sample code in selected answer and recommendation in comment, I have used IReferenceResolver. It turns out that I can't use it, though, but I'll include the code below anyway. The reason why this won't work is because I am trying to bastardize JSON.NET as a quick and dirty cloning tool, so I can't fault it for not suiting my needs. I've since fallen back on my own custom cloning utility, so I no longer need this.
public class ObjectReferenceResolver : Newtonsoft.Json.Serialization.IReferenceResolver
{
readonly Dictionary<object, int> objectDic = new Dictionary<object, int>();
int maxId = 0;
//Called during serialization
public string GetReference(object context, object value)
{
//This method will return the meta $id that you choose. In this example, I am storing
//object references in a dictionary with an incremented ID. If the reference exists, I
//return its ID. Otherwise, I increment the ID and add the reference to the dictionary.
var id = 0;
if (objectDic.ContainsKey(value))
{
id = objectDic[value];
}
else
{
objectDic[value] = maxId++;
}
return id.ToString();
}
//Called during serialization
public bool IsReferenced(object context, object value)
{
//Return whether or not value exists in a reference bank.
//If false, the JSON will return as a full JSON object with "$id": "x"
//If true, the JSON will return "$ref": "x"
return objectDic.ContainsKey(value);
}
//Called during deserialization
public void AddReference(object context, string reference, object value)
{
//This method is called after the deserializer has created a new instance of the
//object. At this time, it's only the initial instance and no properties have been set.
//This method presents a problem because it does not allow you to create the instance or
//retrieve it from a repo and then return it for later use by the reference resolver.
//Therefore, I have to find the existing object by $id, remove it, and then add the new
//object created by the deseralizer. This creates the possibility for two instances of
//the same data object to exist within the running application, so, unfortunately, this
//will not work.
var e = objectDic.First(x => x.Value.ToString() == reference).Key;
objectDic.Remove(e);
objectDic[value] = reference.ParseInt().Value;
}
//Called during deserialization
public object ResolveReference(object context, string reference)
{
//This method retrieves an existing reference by $id and returns it.
var value = objectDic.FirstOrDefault(x => x.Value.ToString() == reference).Key;
return value;
}
}

As per others have recommended, you need a custom IReferenceResolver:
class PersonNameAsIdResolver : IReferenceResolver
{
public void AddReference(object context, string reference, object value)
{
// This method is called during deserialize for $id
}
public string GetReference(object context, object value)
{
// Returns person name as value of $id
return ((Person)value).Name;
}
public bool IsReferenced(object context, object value)
{
// Returns false, so that $id is used, not $ref.
return false;
}
public object ResolveReference(object context, string reference)
{
// This method is called during deserialize for $ref
return null;
}
}
To use that:
var settings = new Newtonsoft.Json.JsonSerializerSettings
{
PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects,
Formatting = Newtonsoft.Json.Formatting.Indented
};
settings.ReferenceResolverProvider = ()=> new PersonNameAsIdResolver();
UPDATE
Answer to the OP's update
AddReference is called while an object is being populated, so it has been too late to replace the object. To be able to find and populate desired object, you need a JsonConverter, which is called before the reference resolver:
class PersonJsonConverter : JsonConverter
{
private readonly PersonNameAsIdResolver _idResolver;
public PersonJsonConverter(PersonNameAsIdResolver idResolver)
{
_idResolver = idResolver;
}
public override bool CanConvert(Type objectType)
=> objectType == typeof(Person);
// Can't write. There's nothing to changing for writing scenario.
public override bool CanWrite => false;
public override object ReadJson(JsonReader reader, Type objectType,
object existingValue, JsonSerializer serializer)
{
var token = JToken.ReadFrom(reader);
if (token.Type == JTokenType.Null)
{
return null;
}
var obj = (JObject)token;
// The code below calls the resolver to find the existing instance.
// This can stop JSON.NET creating a new instance.
Person instance = null;
var #id = obj["$id"].Value<string>();
if (#id != null)
{
instance = (Person)_idResolver.ResolveReference(this, #id);
}
else
{
var #ref = obj["$ref"]?.Value<string>();
if (#ref != null)
{
instance = (Person)_idResolver.ResolveReference(this, #ref);
}
}
// Assuming can't resolve, create a new instance.
if (instance == null)
{
instance = new Person();
}
// This will populate existing Person object if found
serializer.Populate(obj.CreateReader(), instance);
return instance;
}
public override void WriteJson(JsonWriter writer, object value,
JsonSerializer serializer)
{
throw new NotSupportedException();
}
}
And the default serialization settings should look like:
var settings = new Newtonsoft.Json.JsonSerializerSettings
{
PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects,
Formatting = Newtonsoft.Json.Formatting.Indented
};
var idResolver = new PersonNameAsIdResolver();
settings.Converters.Add(new PersonJsonConverter(idResolver));
settings.ReferenceResolverProvider = () => idResolver;

A possible solution would be the following:
Replace PreserveReferncesHandling from Objects to None:
PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.None
Add an Id property in Person class:
public class Person
{
public string Id { get; set; }
public string Name { get; set; }
public Person Mother { get; set; }
}
The complete solution is as follows:
using System;
using Newtonsoft.Json;
namespace ControlJsonId
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("INICIO");
var settings = new Newtonsoft.Json.JsonSerializerSettings
{
PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.None,
Formatting = Newtonsoft.Json.Formatting.Indented,
};
Newtonsoft.Json.JsonConvert.DefaultSettings = () => settings;
var person = new Person
{
Id = Guid.NewGuid().ToString(),
Name = "bob",
Mother = new Person { Id = string.Empty, Name = "jane" }
};
var personJson = JsonConvert.SerializeObject(person);
Console.WriteLine(personJson);
var motherJson = JsonConvert.SerializeObject(person.Mother);
Console.WriteLine(motherJson);
Console.WriteLine("FIN");
Console.ReadKey();
}
}
public class Person
{
public string Id { get; set; }
public string Name { get; set; }
public Person Mother { get; set; }
}
}
The result is:
INICIO
{
"Id": "76d6b5f0-2be8-4d1d-aafe-fe1b4b7d6ae1",
"Name": "bob",
"Mother": {
"Id": "",
"Name": "jane",
"Mother": null
}
}
{
"Id": "",
"Name": "jane",
"Mother": null
}
FIN

Newtonsoft Json converter for json with filters

I am writing converter for json like this:
{
"datatable": {
"data": [
[
"A85002072C",
"1994-11-15",
678.9
]
],
"columns": [
{
"name": "series_id",
"type": "String"
},
{
"name": "date",
"type": "Date"
},
{
"name": "value",
"type": "double"
}
]
},
"meta": {
"next_cursor_id": null
}
}
At the moment my converter looks like this:
public class AbsToModelConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType.Name.Equals("AbsFseModel");
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JArray array = JArray.Load(reader);
return new QuandlAbsModel
{
SeriesId = array[0].ToString(),
Date = array[1].ToObject<DateTime>(),
Value = array[2].ToObject<decimal?>()
};
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var orderItem = value as QuandlAbsModel;
JArray arra = new JArray();
arra.Add(orderItem.SeriesId);
arra.Add(orderItem.Date);
arra.Add(orderItem.Value);
arra.WriteTo(writer);
}
}
It works at the moment, but when i am using filters my json can contain not full data, for example:
"data":[["1994-11-15",678.9]]
And my JsonConverter stops working, because there is no element array[2] and it throws error. Problem is that elements in data array don't have names (i get JSON from web API, so i can't change json at all). Is there any way to make my converter deserialize json with filters?
I have column names in my json after the data table, maybe this will help. But i don't understand how i can use them atm. Any advices?

You don't need a JsonConverter for this.
Define classes to represent the parts of the JSON you need:
class APIResponse
{
public DataTable DataTable { get; set; }
}
class DataTable
{
public object[][] Data { get; set; }
}
Use JsonConvert.DeserializeObject<T>() to deserialize the JSON:
var parsed = JsonConvert.DeserializeObject<APIResponse>(json);
Then get your values:
var rows = parsed.DataTable.Data.Select(r => new QuandLabsModel
{
SeriesId = Convert.ToString(r[0]),
Date = Convert.ToDateTime(r[1]),
Value = Convert.ToDecimal(r[2])
});

JLRishe is correct that your problem is solvable without a custom converter. That's a good approach in many cases. If you're able to insert a translation over the JSON serializer/deserializer, it might be simpler to write, understand, and maintain than a custom JsonConverter. It's similar in spirit to the "serialization proxy pattern" used in the Java world. In essence, you're copying your data to a new serialization-specific object before serializing, and then doing the reverse to re-serialize.
This problem is solvable with a custom converter, and I've written an example to show that it can be done, but do consider using a translation proxy/layer first.
This example is a proof-of-concept; not production-ready code. I made very little effort to defend against malformed input or other errors. Its handling of the different fields/types is also very rudimentary--any changes to the fields/types will require changes to the converter. That sort of brittleness is likely to cause bugs and maintenance headaches over time.
To narrow down the problem a bit, I reduced the original question's sample JSON to its bare minimum:
{
"datatable": {
"data": [
"A85002072C",
"1994-11-15",
678.9
],
"columns": [
{
"name": "series_id"
},
{
"name": "date"
},
{
"name": "value"
}
]
}
}
For reference, here's the C# class definition I'm deserializing to:
public class Model
{
public string SeriesId { get; set; }
public DateTime Date { get; set; }
public Decimal? Value { get; set; }
}
And here's the proof-of-concept converter:
public sealed class ModelConverter : JsonConverter
{
public static readonly ModelConverter Instance = new ModelConverter();
private ModelConverter() {}
public override bool CanConvert(Type objectType) => objectType == typeof(Model);
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var obj = JObject.Load(reader);
var data = (JArray)obj["datatable"]["data"];
var columns = (JArray)obj["datatable"]["columns"];
if (data.Count != columns.Count)
throw new InvalidOperationException("data and columns must contain same number of elements");
var model = new Model();
for (int i = 0; i < data.Count; i++)
{
// A "switch" works well enough so long as the number of fields is finite and small.
// There are smarter approaches, but I've kept the implementation basic
// in order to focus on the core problem that was presented.
switch (columns[i]["name"].ToString())
{
case "series_id":
model.SeriesId = data[i].ToString();
break;
case "date":
model.Date = data[i].ToObject<DateTime>();
break;
case "value":
model.Value = data[i].ToObject<decimal?>();
break;
}
}
return model;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var data = new JArray();
var columns = new JArray();
var model = (Model)value;
// Like the "switch" used in deserialization, these "if" blocks are
// pretty rudimentary. There are better ways, but I wanted to keep
// this proof-of-concept implementation simple.
if (model.SeriesId != default(string))
{
data.Add(model.SeriesId);
columns.Add(new JObject(new JProperty("name", "series_id")));
}
if (model.Date != default(DateTime))
{
data.Add(model.Date.ToString("yyyy-MM-dd"));
columns.Add(new JObject(new JProperty("name", "date")));
}
if (model.Value != default(Decimal?))
{
data.Add(model.Value);
columns.Add(new JObject(new JProperty("name", "value")));
}
var completeObj = new JObject();
completeObj["datatable"] = new JObject();
completeObj["datatable"]["data"] = data;
completeObj["datatable"]["columns"] = columns;
completeObj.WriteTo(writer);
}
}
I wrote a few unit tests to verify the serializer. The tests are based on xUnit.Net:
[Fact]
public void TestDeserializeSampleInputWithAllFields()
{
var json = File.ReadAllText(BasePath + "sampleinput.json");
var obj = JsonConvert.DeserializeObject<Model>(json, ModelConverter.Instance);
Assert.Equal("A85002072C", obj.SeriesId);
Assert.Equal(new DateTime(1994, 11, 15), obj.Date);
Assert.Equal(678.9M, obj.Value);
}
[Fact]
public void TestSerializeSampleInputWithAllFields()
{
var model = new Model
{
SeriesId = "A85002072C",
Date = new DateTime(1994, 11, 15),
Value = 678.9M,
};
var expectedJson = File.ReadAllText(BasePath + "sampleinput.json");
Assert.Equal(expectedJson, JsonConvert.SerializeObject(model, Formatting.Indented, ModelConverter.Instance));
}
And to prove that the serializer works without all fields present:
{
"datatable": {
"data": [
"B72008039G",
543.2
],
"columns": [
{
"name": "series_id"
},
{
"name": "value"
}
]
}
}
[Fact]
public void TestDeserializeSampleInputWithNoDate()
{
var json = File.ReadAllText(BasePath + "sampleinput_NoDate.json");
var obj = JsonConvert.DeserializeObject<Model>(json, ModelConverter.Instance);
Assert.Equal("B72008039G", obj.SeriesId);
Assert.Equal(default(DateTime), obj.Date);
Assert.Equal(543.2M, obj.Value);
}
[Fact]
public void TestSerializeSampleInputWithNoDate()
{
var model = new Model
{
SeriesId = "B72008039G",
Value = 543.2M,
};
var expectedJson = File.ReadAllText(BasePath + "sampleinput_NoDate.json");
Assert.Equal(expectedJson, JsonConvert.SerializeObject(model, Formatting.Indented, ModelConverter.Instance));
}

Deserializing Json array with variable names first using C# Json.NET

I'm getting an irregular JSON array from the Census Bureau's public api.
The variable names are all in the first element, and I'm having trouble deserializing it.
http://api.census.gov/data/2014/pep/agesex?get=AGE,POP,SEX&for=us:*&DATE=7
gives me JSON like this:
[["AGE","POP","SEX","DATE","us"],
["0","3948350","0","7","1"],
["1","3962123","0","7","1"],
["2","3957772","0","7","1"],
["3","4005190","0","7","1"],
["4","4003448","0","7","1"],
["5","4004858","0","7","1"],
["6","4134352","0","7","1"],
["7","4154000","0","7","1"]]
I can successfully deserialize this using:
var test1 = JsonConvert.DeserializeObject<String[][]>(jsonStr);
However, I'm trying to deserialize it to a class like this:
public class TestClass
{
public string AGE { get; set; }
public string POP { get; set; }
public string SEX { get; set; }
public string DATE { get; set; }
public string us { get; set; }
}
I'm trying to do this:
var test2 = JsonConvert.DeserializeObject<TestClass[]>(jsonStr);
But I'm getting the following exception:
An exception of type 'Newtonsoft.Json.JsonSerializationException'
occurred in Newtonsoft.Json.dll but was not handled in user code
Additional information: Cannot create and populate list type
TestClass. Path '[0]', line 1, position
2.

There's two parts to this.
First is turning the JSON in to data usable in C#, and the second is turning that data in to nice objects.
Here's a working dotNetFiddle.net example of the following code: https://dotnetfiddle.net/Cr0aRL
Each row in your JSON is made up of an array of strings.
So that's an array of an array of strings.
In C# that can be written as string[][].
So to turn the JSON in to usable data with JSON.Net you can do:
var json = "[[\"AGE\",\"POP\",\"SEX\",\"DATE\",\"us\"],[\"0\",\"3948350\",\"0\",\"7\",\"1\"],[\"1\",\"3962123\",\"0\",\"7\",\"1\"],[\"2\",\"3957772\",\"0\",\"7\",\"1\"],[\"3\",\"4005190\",\"0\",\"7\",\"1\"],[\"4\",\"4003448\",\"0\",\"7\",\"1\"],[\"5\",\"4004858\",\"0\",\"7\",\"1\"],[\"6\",\"4134352\",\"0\",\"7\",\"1\"],[\"7\",\"4154000\",\"0\",\"7\",\"1\"]]";
var rawData = JsonConvert.DeserializeObject<string[][]>(json);
Next up is is turning that data in to objects.
The first row is the header, containing the column names, so we want to grab that, and then figure out the column index for each column name.
var headerRow = rawData.First();
var ageIndex = Array.IndexOf(headerRow, "AGE");
var popIndex = Array.IndexOf(headerRow, "POP");
var sexIndex = Array.IndexOf(headerRow, "SEX");
var dateIndex = Array.IndexOf(headerRow, "DATE");
var usIndex = Array.IndexOf(headerRow, "us");
Now we have the indexes, we need to take each row, and convert it in to the appropriate object. I've used LINQ for this as it's very good at representing data processing in a clear way.
var testData = rawData
.Skip(1) //The first row is a header, not data
.Select(dataRow => new TestClass()
{
AGE = dataRow[ageIndex],
POP = dataRow[popIndex],
SEX = dataRow[sexIndex],
DATE = dataRow[dateIndex],
us = dataRow[usIndex]
});
Finally a bit of testing, to make sure you have the data you're expecting.
//Get the second data row as an example
var example = testData.Skip(1).First();
//Output example POP to check value
Console.WriteLine(example.POP);
Everything above is very manual.
You have to know what headers you expect, then you manually find the indexes, then you manually map the rows to objects.
It's quite possible for a simple use case that doing that is fine. But in larger and/or more complex systems you might want/need to automate those steps.
Automating those steps is possible, but is beyond the scope of this answer as how you approach it can depend on a lot of different factors.

You could make a custom JsonConverter to handle this conversion during deserialization. The conversion code is really not much different than other answers here, except that it is encapsulated into a separate class so that you don't muddy up your main code with the conversion details. From the point of view of your main code it "just works".
Here is how to write the converter:
public class TestClassArrayConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(TestClass[]));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JArray table = JArray.Load(reader);
TestClass[] items = new TestClass[table.Count - 1];
for (int i = 1; i < table.Count; i++)
{
JArray row = (JArray)table[i];
items[i - 1] = new TestClass
{
AGE = (string)row[0],
POP = (string)row[1],
SEX = (string)row[2],
DATE = (string)row[3],
us = (string)row[4]
};
}
return items;
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
And here is how you would use it:
var test2 = JsonConvert.DeserializeObject<TestClass[]>(jsonStr, new TestClassArrayConverter());
Fiddle: https://dotnetfiddle.net/68Q0KT

You have to do the processing on your own, as there is no way the json deserializer can know, how to put the values into the respecitve variables.
If you know, this will be exactly this structure, you could for instance add an appropriate constructor
public TestClass(string[] values) {
AGE = values[0];
...
}
to your class. Then serialize your result to array of arrays of string and then pass the inner arrays to your constructor.
var t1 = JsonConvert.DeserializeObject<string[][]>(jsonStr);
//skip the first entry, as this contains the headers
var t2 = t1.Skip(1).Select(x=> new TestClass(x));
If your structure varies, you'll have to write some more complicated mapping code.

You will have to do some custom mapping as your Json does not have any naming conventions so you will have to work with the data in array and index formats. This will work:
var jsonStr = "[[\"AGE\",\"POP\",\"SEX\",\"DATE\",\"us\"], [\"0\",\"3948350\",\"0\",\"7\",\"1\"], [\"1\",\"3962123\",\"0\",\"7\",\"1\"], [\"2\",\"3957772\",\"0\",\"7\",\"1\"], [\"3\",\"4005190\",\"0\",\"7\",\"1\"], [\"4\",\"4003448\",\"0\",\"7\",\"1\"], [\"5\",\"4004858\",\"0\",\"7\",\"1\"], [\"6\",\"4134352\",\"0\",\"7\",\"1\"], [\"7\",\"4154000\",\"0\",\"7\",\"1\"]]";
var test2 = JsonConvert.DeserializeObject<string[][]>(jsonStr);
var test3 = test2.Select(x => new TestClass()
{
AGE = x[0].ToString(),
POP = x[1].ToString(),
SEX = x[2].ToString(),
DATE = x[3].ToString(),
us = x[4].ToString()
}).ToList();

//test Case
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
namespace ApiController.Test
{
[TestClass]
public class DownloadIrregularJsonStringObjects
{
string ApiKey => "YourPersonalCensusKey";
/// <summary>
/// You have to get your own ApiKey from the Census Website
/// </summary>
[TestMethod]
public void TestGetItem()
{
string url = $"http://api.census.gov/data/timeseries/healthins/sahie?get=NIC_PT,NAME,NUI_PT&for=county:*&in=state:*&time=2015&key={YourPersonalCensusKey}";
string expected = "Autauga County, AL";
IList<HealthData> actual = ApiController.DownloadIrregularJsonStringObjects.GetCensusHealthData(url);
Assert.AreEqual(actual[0].NAME, expected);
}
}
}
///Actual Assembly
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
namespace ApiController
{
public class DownloadIrregularJsonStringObjects
{
public static IList<HealthData> GetCensusHealthData(string url)
{
var json = GetData(url);
var rawData = JsonConvert.DeserializeObject<string[][]>(json);
var headerRow = rawData.First();
var nic_pt_Index = Array.IndexOf(headerRow, "NIC_PT");
var name_Index = Array.IndexOf(headerRow, "NAME");
var nui_pt_Index = Array.IndexOf(headerRow, "NUI_PT");
IList<HealthData> retVal = new List<HealthData>();
foreach (var r in rawData.Skip(1))
{
HealthData dataRow = new HealthData();
dataRow.NIC_PT = r[nic_pt_Index];
dataRow.NAME = r[name_Index];
dataRow.NUI_PT = r[nui_pt_Index];
retVal.Add(dataRow);
}
return retVal;
}
private static string GetData(string url)
{
using (var w = new WebClient())
{
var jsonData = string.Empty;
jsonData = w.DownloadString(url);
return jsonData;
}
}
}
public class HealthData
{
public string NIC_PT { get; set; }
public string NAME { get; set; }
public string NUI_PT { get; set; }
}
}

Serialise list of C# custom objects

I am receiving a JSON string from API that is structured in not a good way to be handled in the App.
I chose to create a custom serialiser for this JSON data (rather then having two different classes for data received and used in app).
Following a lot of tutorials I managed to put together a custom serialiser for a single object. However, I need to work with lists of these objects (there will be more different data that will come in these weird lists, that needs custom handling).
Is there a built in way I can set my custom serialiser to work with each object in the list? Or do I need to split the JSON object manually, and feed chunks of it to custom serialiser?
Any other suggestions how to handle this situation is appreciated.
User class:
[JsonConverter(typeof(UserSerializer))]
public class User
{
public int id;
public string displayName;
public string email;
public int total_points;
public float total_values;
}
The deserialiser:
public override object ReadJson(JsonReader reader, Type objectType,
object existingValue, JsonSerializer serializer)
{
JObject jObjectRoot = JObject.Load(reader);
var root = jObjectRoot.Properties().ToList();
JObject jObjectData = JObject.Load(root[2].Value.CreateReader());
var data = jObjectData.Properties().ToList();
return new User {
id = (int)root[1].Value,
displayName = (string)data[0].Value,
email = (string)data[1].Value,
total_points = (int)data[2].Value,
total_values = (float)data[3].Value
};
}
UPDATE:
Also the code that parses the json string to single user object:
public static void ProcessJSON(string json)
{
User user = JsonConvert.DeserializeObject<User>(json);
}
And the JSON itself:
[
{
"type": "users",
"id": 1,
"attr": {
"display_name": "user2",
"email": "user2#email.com",
"total_points": 4,
"total_values": 32.34
},
"relationships": {
"points_received": {
"links": {
"self": "tipjar/users/1/relationships/points",
"related": "tipjar/users/1/points"
}
},
"points_given": {
"links": {
"self": "tipjar/users/1/relationships/awarded",
"related": "tipjar/users/1/awarded"
}
}
}
}
]
Thanks

You can get the list of user objects without a custom converter like this:
var userList = JArray.Parse(json)
.Select(t => new User()
{
id = int.Parse(t["id"].ToString()),
displayName = t["attr"]["display_name"].ToString(),
email = t["attr"]["email"].ToString(),
total_points = int.Parse(t["attr"]["total_points"].ToString()),
total_values = float.Parse(t["attr"]["total_values"].ToString()),
}).ToList();

public static void ProcessJSON(string json)
{
User u = new User();
var test = JsonConvert.DeserializeObject(json);
if (test.GetType() == typeof(User))
u = (User)test;
}
Not 100% on how the serialize works, but this seemed to have worked on the test app I made.
It might be a bit silly. But you could test on the different types of objects returned...

Is there a way in Json.NET serialization to distinguish between "null because not present" and "null because null"?

I'm working in an ASP.NET webapi codebase where we rely heavily on the automatic support for JSON deserialization of message bodies into .NET objects via JSON.NET.
As part of building out patch support for one of our resources, I'd very much like to distinguish between an optional property in the JSON object that's not present, vs. that same property that's explicitly to null. My intention is to use the first for "don't change what's there" vs. "delete this thing."
Does anyone know if it's possible to mark up my C# DTOs so that when they're deserialized that JSON.NET can tell me which case it was? Right now they're just come up as null, and I can't tell why.
Conversely, if anyone can come up with a better design that doesn't require me to do it this way while still supporting the patch verb, I'd love to hear your proposal.
As a concrete example, consider this payload that would be passed to put:
{
"field1": "my field 1",
"nested": {
"nested1": "something",
"nested2": "else"
}
}
Now, if I just wanted to update field1, I should be able to send this as an HTTP patch:
{
"field1": "new field1 value"
}
and the nested values would remain untouched. However, if I sent this:
{
"nested": null
}
I want to know this means I should explicitly remove the nested data.

If you use Json.Net's LINQ-to-JSON API (JTokens, JObjects, etc.) to parse the JSON, you can tell the difference between a null value and a field that simply doesn't exist in the JSON. For example:
JToken root = JToken.Parse(json);
JToken nested = root["nested"];
if (nested != null)
{
if (nested.Type == JTokenType.Null)
{
Console.WriteLine("nested is set to null");
}
else
{
Console.WriteLine("nested has a value: " + nested.ToString());
}
}
else
{
Console.WriteLine("nested does not exist");
}
Fiddle: https://dotnetfiddle.net/VJO7ay
UPDATE
If you're deserializing into concrete objects using Web API, you can still use the above concept by creating a custom JsonConverter to handle your DTOs. The catch is that there needs to be a place on your DTOs to store the field status during deserialization. I would suggest using a dictionary-based scheme like this:
enum FieldDeserializationStatus { WasNotPresent, WasSetToNull, HasValue }
interface IHasFieldStatus
{
Dictionary<string, FieldDeserializationStatus> FieldStatus { get; set; }
}
class FooDTO : IHasFieldStatus
{
public string Field1 { get; set; }
public BarDTO Nested { get; set; }
public Dictionary<string, FieldDeserializationStatus> FieldStatus { get; set; }
}
class BarDTO : IHasFieldStatus
{
public int Num { get; set; }
public string Str { get; set; }
public bool Bool { get; set; }
public decimal Dec { get; set; }
public Dictionary<string, FieldDeserializationStatus> FieldStatus { get; set; }
}
The custom converter would then use above LINQ-to-JSON technique to read the JSON for the object being deserialized. For each field in the target object, it would add an item to that object's FieldStatus dictionary indicating whether the field had a value, was explicitly set to null or did not exist in the JSON. Here is what the code might look like:
class DtoConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType.IsClass &&
objectType.GetInterfaces().Any(i => i == typeof(IHasFieldStatus)));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var jsonObj = JObject.Load(reader);
var targetObj = (IHasFieldStatus)Activator.CreateInstance(objectType);
var dict = new Dictionary<string, FieldDeserializationStatus>();
targetObj.FieldStatus = dict;
foreach (PropertyInfo prop in objectType.GetProperties())
{
if (prop.CanWrite && prop.Name != "FieldStatus")
{
JToken value;
if (jsonObj.TryGetValue(prop.Name, StringComparison.OrdinalIgnoreCase, out value))
{
if (value.Type == JTokenType.Null)
{
dict.Add(prop.Name, FieldDeserializationStatus.WasSetToNull);
}
else
{
prop.SetValue(targetObj, value.ToObject(prop.PropertyType, serializer));
dict.Add(prop.Name, FieldDeserializationStatus.HasValue);
}
}
else
{
dict.Add(prop.Name, FieldDeserializationStatus.WasNotPresent);
}
}
}
return targetObj;
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
The above converter will work on any object that implements the IHasFieldStatus interface. (Note that you do not need to implement the WriteJson method in the converter unless you intend to do something custom on serialization as well. Since CanWrite returns false, the converter will not be used during serialization.)
Now, to use the converter in Web API, you need to insert it into the configuration. Add this to your Application_Start() method:
var config = GlobalConfiguration.Configuration;
var jsonSettings = config.Formatters.JsonFormatter.SerializerSettings;
jsonSettings.C‌​onverters.Add(new DtoConverter());
If you prefer, you can decorate each DTO with a [JsonConverter] attribute like this instead of setting the converter in the global config:
[JsonConverter(typeof(DtoConverter))]
class FooDTO : IHasFieldStatus
{
...
}
With the converter infrastructure in place, you can then interrogate the FieldStatus dictionary on the DTO after deserialization to see what happened for any particular field. Here is a full demo (console app):
public class Program
{
public static void Main()
{
ParseAndDump("First run", #"{
""field1"": ""my field 1"",
""nested"": {
""num"": null,
""str"": ""blah"",
""dec"": 3.14
}
}");
ParseAndDump("Second run", #"{
""field1"": ""new field value""
}");
ParseAndDump("Third run", #"{
""nested"": null
}");
}
private static void ParseAndDump(string comment, string json)
{
Console.WriteLine("--- " + comment + " ---");
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.Converters.Add(new DtoConverter());
FooDTO foo = JsonConvert.DeserializeObject<FooDTO>(json, settings);
Dump(foo, "");
Console.WriteLine();
}
private static void Dump(IHasFieldStatus dto, string indent)
{
foreach (PropertyInfo prop in dto.GetType().GetProperties())
{
if (prop.Name == "FieldStatus") continue;
Console.Write(indent + prop.Name + ": ");
object val = prop.GetValue(dto);
if (val is IHasFieldStatus)
{
Console.WriteLine();
Dump((IHasFieldStatus)val, " ");
}
else
{
FieldDeserializationStatus status = dto.FieldStatus[prop.Name];
if (val != null)
Console.Write(val.ToString() + " ");
if (status != FieldDeserializationStatus.HasValue)
Console.Write("(" + status + ")");
Console.WriteLine();
}
}
}
}
Output:
--- First run ---
Field1: my field 1
Nested:
Num: 0 (WasSetToNull)
Str: blah
Bool: False (WasNotPresent)
Dec: 3.14
--- Second run ---
Field1: new field value
Nested: (WasNotPresent)
--- Third run ---
Field1: (WasNotPresent)
Nested: (WasSetToNull)
Fiddle: https://dotnetfiddle.net/xyKrg2

Looking through the Json.NET source, I found that it supports populating bool properties with a suffix of "Specified" to indicate whether or not the property was included in the data:
class MyClass
{
public string Field1 { get; set; }
public Nested Nested { get; set; }
public bool NestedSpecified { get; set; }
}
class Nested
{
public string Nested1 { get; set; }
public string Nested2 { get; set; }
}
Input:
{
"field1": "my field 1",
"nested": {
"nested1": "something",
"nested2": "else"
}
}
Resulting instance:
MyClass { Field1="my field 1", Nested=Nested { Nested1="something", Nested2="else" }, NestedSpecified=true }
Input:
{
"field1": "new field1 value"
}
Resulting instance:
MyClass { Field1="new field1 value", Nested=null, NestedSpecified=false }
Input:
{
"nested": null
}
Resulting instance:
MyClass { Field1=null, Nested=null, NestedSpecified=true }
I can't find this functionality in the Json.NET documentation but it looks like it has been there since 2010.

You could add some metadata to your JSON objects and (most likely) DTOs. It would require additional processing, but is pretty transparent and unambiguously accomplishes what you need (assuming you can name the new field such that you know it won't collide with actual data).
{
"deletedItems": null,
"field1": "my field 1",
"nested": {
"deletedItems": null,
"nested1": "something",
"nested2": "else"
}
}
{
"deletedItems": "nested",
"field1": "new value",
"nested": null
}
Alternatively, you could add an "isDeleted" property per field if your object model accommodates that better, but that sounds like a lot more work than a list of deleted fields.

I don't want to hijack this question but I posted a slightly different approach to this problem here: https://stackoverflow.com/a/31489835/1395758.
The approach is to replace the fields in your deserializable type with a struct that will automatically keep track of values (even null) through an IsSet property.

The most elegant solution I came up with came to me in an epiphany:
using System;
using System.Collections.Generic;
using System.Linq;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace testJsonDeserializer
{
class Program
{
static void Main(string[] args)
{
// this operator has the password set to meow.
Operator originalOperator = new Operator
{
OperatorGuid = Guid.Parse("3bb1dc84-2963-4921-a567-fb2e7475623d"),
UserName = "noortje#peterhuppertz.net",
Password = "meow",
PropertyThatWillBeNulled = "noortje#peterhuppertz.net",
};
// this json EXPLICITLY sets the PropertyThatWillBeNulled to null, but omits the Password property, making it null IMPLICITLY.
string json =
"{ \"OperatorGuid\":\"3bb1dc84-2963-4921-a567-fb2e7475623d\", \"UserName\": \"noortje#peterhuppertz.net\", \"Email\": null }";
// What a PATCH would want for the target object is to leave implicit Nulls unchanged, but explicit nulls set to null.
Operator patchedOperator = JsonConvert.DeserializeObject<Operator>(json);
// At this stage, our patched operator has the password set to null. We do not want that; we want to keep whatever is stored in originalOperator
Operator opToStore = MapJsonToOperator(patchedOperator, originalOperator, json);
Console.WriteLine("Our patched operator:");
Console.WriteLine($"Guid: {opToStore.OperatorGuid}");
Console.WriteLine($"UserName: {opToStore.UserName}");
Console.WriteLine($"Password: {opToStore.Password}");
Console.WriteLine($"Email: {opToStore.PropertyThatWillBeNulled}");
Console.ReadKey();
}
private static Operator MapJsonToOperator(Operator source, Operator original, string json)
{
Operator result = new Operator
{
OperatorGuid = source.OperatorGuid,
UserName = source.UserName != null
// we check if the source property has a value, if so, we use that value.
? source.UserName
// if it doesn't, we check the Json to see if the value is in there, explicitly set to NULL. If it is, we set it to NULL as well
: (IsNullValueExplicit(json, "UserName") ? null
// if it is not in the json (making it implicitly null), we just leave the value as it was.
: original.UserName),
PropertyThatWillBeNulled = source.PropertyThatWillBeNulled != null
? source.PropertyThatWillBeNulled
: (IsNullValueExplicit(json, "Email") ? null : original.PropertyThatWillBeNulled),
Password = source.Password != null
? source.Password
: (IsNullValueExplicit(json, "Password") ? null : original.Password),
};
return result;
}
static bool IsNullValueExplicit(string json, string fieldName)
{
JToken outer = JToken.Parse(json);
JObject inner = outer.Value<JObject>();
List<string> keys = inner.Properties().Select(p => p.Name).ToList();
return keys.Contains(fieldName);
}
}
public class Operator
{
public Guid OperatorGuid { get; set; }
public string UserName { get; set; }
public string Password { get; set; }
public string PropertyThatWillBeNulled { get; set; }
}
}
I know, there are a lot of comments in here. Maybe I overexplained... but I thought I'd err on the side of caution.