We Keep Coding

How to parse a complicated nested JSON string in C#?

I have an ugly JSON string that is getting returned from an API that looks like this (this is the result of Console.Write on the string):
{"d":"\"\\\"\\\\\\\"[{\\\\\\\\\\\\\\\"foo\\\\\\\\\\\\\\\":15,\\\\\\\\\\\\\\\"bar\\\\\\\\\\\\\\\":null}]\\\\\\\"\\\"\\n\""}
I am trying to parse this into a C# object in the simplest way possible, so I can access properties like foo and bar. But I am having a difficult time doing this.
I have tried parsing it a number of ways, including:
// code to get the response string
client.Headers.Add(HttpRequestHeader.ContentType, "application/json");
var serializedData = "{data: 'data'}";
var responseString = client.UploadString(url, "POST", serializedData);
// parse the response string
dynamic obj = Newtonsoft.Json.JsonConvert.DeserializeObject(jsonString);
This allows me to access the value of d, which is the actual string I need to parse. I then tried to parse that separately using JArray.Parse(obj["d"]), but I get an error saying that obj["d"] is not an array.
Unfortunately, I have no access to the API itself so can't modify how it's serializing the data it's returning.
Any suggestions?

You can replace all New Line, Backslash, Double quotes to format the JSON
var formattedJson = jsonString.Replace(#"\n", string.Empty)
.Replace(#"\", string.Empty)
.Replace("\"\"", string.Empty);
Console.WriteLine(formattedJson);
OUTPUT
{
"d": [
{
"foo": 15,
"bar": null
}
]
}
Convert to JArray.
var jArray = JArray.Parse(JObject.Parse(formattedJson)["d"].ToString());
Console.WriteLine($"{jArray[0]["foo"]} {jArray[0]["bar"]}");
OUTPUT
15

The problem is that the value of "d" is a string representing a string representing a string ... representing an array. You could call it JSON serialization "inception".
The way to deal with this is to deserialize the value corresponding number of times. If you're sure that the value is never going to be an actual string, you could do it like this, without having to know how many times the value was serialized:
var myObject = JObject.Parse(s);
var d = myObject["d"];
while(d.Type == JTokenType.String)
d = JToken.Parse(d.ToObject<string>());
myObject["d"] = d;
After this procedure myObject represents this data:
{
"d": [
{
"foo": 15,
"bar": null
}
]
}

Replacing escape characters in fine however I would not rely on the console.write command as the definitive output to examine. Here are a couple of other ways: -
Use Postman to make the API call so you can see the raw result. This will (hopefully) show it in an easy to read format that you can then define your class to deserialise to.
Write the raw response to a “.json” file. Open that file in a good editor (such as VS Code or VS itself) to see how the data is actually structured when it is received.
On a side note I would recommend using RestSharp to do the REST calls and Newtonsoft.Json to do the serialising/deserialising.

Convert string into a valid JSON in c#

In the code snippet below, the JSON string in the commented out jsonString variable is valid while the uncommented out one causes JObject.Parse to throw a JsonReaderException with the message:
After parsing a value an unexpected character was encountered: e. Path 'Key', line 1, position 15.
var jsonString = "{\"Key\":\"Value \"extra\" \"}";
//var jsonString = "{\"Key\":\"Value \\\"extra\\\" \"}";
JObject.Parse(jsonString);
Are there any methods available in Newtonsoft.Json or elsewhere that can transform a JSON string to make it valid?

No, because NewtonSoft cannot guess what you want. E.g. is extra a new key and did you just ommit a comma or is it part of the previous value, or is it just something that can be ignored. It would be better to have the thing you are consuming the json from construct valid json.

Using Regex might help you to resolve the existing JSON you have. If you can control how subsequent JSON is generated, you really should fix it at that point.
This solution counts the value as existing from the first " after a "key":, through to the last " before a , or a }, and then it reserializes the value to ensure that it is correctly escaped. If it finds ",, it expects it to be followed by another key ("key":). This is in an attempt to avoid red herrings (i.e. {"key": "test "," value"}) which might otherwise confuse it.
private static string FixJson(string json)
{
var regex = new Regex("\"(?<key>.*?)\"\\W?:\\W?\"(?<value>.*?)\"(?=,\".*?\"\\W?:|}$)");
return regex.Replace(json, new MatchEvaluator(m => {
var key = m.Groups["key"].Value;
var val = m.Groups["value"].Value;
return string.Format("\"{0}\":{1}", key, JsonConvert.SerializeObject(val));
}));
}
Disclaimer: It's a regular expression, it's not foolproof, and if your JSON is more broken than you have indicated, it will probably spit out broken JSON, or incorrect values, so use it at your own risk.
Try it online

Is "[]" valid JSON?

I'm having troubles de-serializing this JSON string using JSON.NET (note the quotes):
"[]"
Depending on which JSON validation website you go to, this is valid JSON (jsonlint for example says it is).
The JSON.NET code:
void Main()
{
string json = "\"[]\"";
var x = JsonConvert.DeserializeObject<User[]>(json);
Console.WriteLine(x);
}
// Define other methods and classes here
public class User
{
public string Id { get; set; }
public int Age { get; set; }
}
The exception
Error converting value "[]" to type 'UserQuery+User[]'. Path '', line 1, position 4.
Is there a way of forcing JSON.NET to parse this?

Part 1: Is "[]" valid JSON?
There are several documents and standards on JSON, and hundreds of parsers; and some of them suppose that JSON can only be object {} or an array [], but some allow single values like strings, numbers to be used as JSON.
Read this article, it widely describes this problem.
What is the minimum valid JSON?
This dispute on JSON validity is another question. In your case, it doesn't matter, because...
Part 2: why your code isn't working.
Even if we allow non-objects \ non-arrays to be valid JSON, then your JSON represents a single string equal to "[]". It could be anything else, not brackets, it is not an array notation, but just two symbols "[" and "]".
However, you try to parse this JSON as an array of objects, which will anyway result into error.
In other words, even if it is a valid JSON, then it is a valid JSON string, not JSON array.
var str1 = JSON.parse("\"[]\""),
str2 = JSON.parse("\"could be anything else, not brackets\""),
arr = JSON.parse("[]");
console.log(typeof str1);
console.log(typeof str2);
console.log(typeof arr);
var str1_s = JSON.stringify([]);
console.log("Valid JSON of an empty array: " + str1_s);
var arr_s = JSON.stringify("[]");
console.log("Partly valid JSON of a string '[]': " + arr_s);
Part 3: what should you do
The best idea - stop using invalid JSON as input. Tell whoever gave you this JSON that it is invalid JSON array and you cannot use it. You would be able to deserialize a JSON into your array of User if it was correct just like you use it:
string json = "[]";
var x = JsonConvert.DeserializeObject<User[]>(json);
Console.WriteLine(x);
If this JSON is provided from 3rd party services and you can do nothing about that, then you need to tidy it up and make it valid. Yeah, unfortunately, sometimes it happens.
How? It depends on what is your value when there ARE objects (users).
It may be a JSON-serialized JSON-string (double-serialized) like this, and then you need to deserialize a string, and then deserialize an array.
Or it can just have two odd quotes in the beginning and the end, and you can just remove them.

It is valid JSON, but the deserializer failes because the datatypes do not match.
"[]"
Is a string, so the deserializer wants to serialize it to a string.
[]
Is an empty array. So, in short, this should work:
string json = "[]";
var x = JsonConvert.DeserializeObject<User[]>(json);
Console.WriteLine(x);

Custom Json serialization

I have a list of KeyPairValue wich I serialize in Json Object using JavaScriptSerializer. The output of this operation give me something like this :
[{"Key":"A","Value":"ValueA"},{"Key":"D","Value":"ValueD"}]
I'ld like to get rid of those "" around the property name so it could look like this :
[{ Key:"A", Value:"ValueA"},{ Key:"D", Value:"ValueD"}]
Is there a way to achieve this or if what i'm looking for is just not a Json serialization ?

You can achieve it with Json.Net
StringWriter str = new StringWriter();
JsonTextWriter writer = new JsonTextWriter(str);
writer.QuoteName = false; //<-- This is the trick
JsonSerializer jsonSer = new JsonSerializer();
jsonSer.Serialize(writer, new { ID = 1, Name = "Jack" });
string jsonstr = str.ToString();
Output is {ID:1,Name:"Jack"}

As I know it is a must requirement by JSON to embedd your keys in "". This is because JSON is actually a transport format for JavaScript objects and therefore it has some specifics.

The RFC specifies that a name is a string, and it also specifies that a string must be wrapped in quotation marks.
RFC 4627
It's worth noting the different types that values can be and that some of the types don't have to be wrapped in quotes. You can find this in the spec as well.

Deserializing Client-Side AJAX JSON Dates

Given the following JSON Date representation:
"\/Date(1221644506800-0700)\/"
How do you deserialize this into it's JavaScript Date-type form?
I've tried using MS AJAX JavaScrioptSerializer as shown below:
Sys.Serialization.JavaScriptSerializer.deserialize("\/Date(1221644506800-0700)\/")
However, all I get back is the literal string date.

Provided you know the string is definitely a date I prefer to do this :
new Date(parseInt(value.replace("/Date(", "").replace(")/",""), 10))

Bertrand LeRoy, who worked on ASP.NET Atlas/AJAX, described the design of the JavaScriptSerializer DateTime output and revealed the origin of the mysterious leading and trailing forward slashes. He made this recommendation:
run a simple search for "\/Date((\d+))\/" and replace with "new Date($1)" before the eval
(but after validation)
I implemented that as:
var serializedDateTime = "\/Date(1271389496563)\/";
document.writeln("Serialized: " + serializedDateTime + "<br />");
var toDateRe = new RegExp("^/Date\\((\\d+)\\)/$");
function toDate(s) {
if (!s) {
return null;
}
var constructor = s.replace(toDateRe, "new Date($1)");
if (constructor == s) {
throw 'Invalid serialized DateTime value: "' + s + '"';
}
return eval(constructor);
}
document.writeln("Deserialized: " + toDate(serializedDateTime) + "<br />");
This is very close to the many of the other answers:
Use an anchored RegEx as Sjoerd Visscher did -- don't forget the ^ and $.
Avoid string.replace, and the 'g' or 'i' options on your RegEx. "/Date(1271389496563)//Date(1271389496563)/" shouldn't work at all.

A JSON value is a string, number, object, array, true, false or null. So this is just a string. There is no official way to represent dates in JSON. This syntax is from the asp.net ajax implementation. Others use the ISO 8601 format.
You can parse it like this:
var s = "\/Date(1221644506800-0700)\/";
var m = s.match(/^\/Date\((\d+)([-+]\d\d)(\d\d)\)\/$/);
var date = null;
if (m)
date = new Date(1*m[1] + 3600000*m[2] + 60000*m[3]);

The regular expression used in the ASP.net AJAX deserialize method looks for a string that looks like "/Date(1234)/" (The string itself actually needs to contain the quotes and slashes). To get such a string, you will need to escape the quote and back slash characters, so the javascript code to create the string looks like "\"\/Date(1234)\/\"".
This will work.
Sys.Serialization.JavaScriptSerializer.deserialize("\"\\/Date(1221644506800)\\/\"")
It's kind of weird, but I found I had to serialize a date, then serialize the string returned from that, then deserialize on the client side once.
Something like this.
Script.Serialization.JavaScriptSerializer jss = new Script.Serialization.JavaScriptSerializer();
string script = string.Format("alert(Sys.Serialization.JavaScriptSerializer.deserialize({0}));", jss.Serialize(jss.Serialize(DateTime.Now)));
Page.ClientScript.RegisterStartupScript(this.GetType(), "ClientScript", script, true);

For those who don't want to use Microsoft Ajax, simply add a prototype function to the string class.
E.g.
String.prototype.dateFromJSON = function () {
return eval(this.replace(/\/Date\((\d+)\)\//gi, "new Date($1)"));
};
Don't want to use eval? Try something simple like
var date = new Date(parseInt(jsonDate.substr(6)));
As a side note, I used to think Microsoft was misleading by using this format. However, the JSON specification is not very clear when it comes to defining a way to describe dates in JSON.

Actually, momentjs supports this kind of format, you might do something like:
var momentValue = moment(value);
momentValue.toDate();
This returns the value in a javascript date format

The big number is the standard JS time
new Date(1221644506800)
Wed Sep 17 2008 19:41:46 GMT+1000 (EST)