Deserialize JSON to list with unknown object name in C# - c#

I want to deserialize following JSON.
The problem is that the objects "ANDE" & "DAR" can change.
Means the objects are unknown and change depending on the JSON i wanna deserialize.
About 8000 different objects (ANDE, DAR, ...) need to be deserialized.
{"ANDE":
{"chart":[
{"date":"20180914","minute":"09:30"},{"date":"20180914","minute":"13:30"}]},
"DAR":
{"chart":[
{"date":"20180914","minute":"09:30"},{"date":"20180914","minute":"13:30"}]}}
I get the data by HTTP response and want to put into a List:
HttpResponseMessage response = client.GetAsync(API_PATH).GetAwaiter().GetResult();
List historicalDataList = response.Content.ReadAsAsync<List<HistoricalDataResponse>>().GetAwaiter().GetResult();
The HistoricalDataResponse class looks like:
public class HistoricalDataResponse
{
public string date { get; set; }
public string minute { get; set; }
}
How can i deserialize this kind of JSON with unknown objects in C#?
Any help is highly appreciated.


Then you should use a dynamic variable:
dynamic ReturnValue = JsonConvert.DeserializeObject(jsonstring);
note that as in dynamic objects, properties are determined after being assigned in runtime, so you will not get a drop down menu in design time, and also as its properties are unknown in design time, and property you test in design time even if its not correct, you wont get an error, and you will get the error in runtime when it is assigned.
dynamic ReturnValue = JsonConvert.DeserializeObject(jsonstring);
try
{
var a = ReturnValue.ANDE; // will work if it has ANDE property.
// do what you would do with ANDE
}
catch{}
try
{
var a = ReturnValue.DAR; // will work if it has DAR property.
// do what you would do with DAR
}
catch{}


Use a dictionary with string as key type :
void Main()
{
var client = new HttpClient();
HttpResponseMessage response = client.GetAsync("url").GetAwaiter().GetResult();
var json = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
var result = JsonConvert.DeserializeObject<Dictionary<string, DateResponse>>(json);
foreach (var element in result)
{
var key = element.Key; // ANDE
foreach (var item in element.Value.Chart)
{
var date = item.date;
var minute = item.minute;
}
}
}
public class DateResponse{
public List<HistoricalDataResponse> Chart { get; set; }
}
public class HistoricalDataResponse
{
public string date { get; set; }
public string minute { get; set; }
}

Related

How to get all not exist keys after JsonConvert DeserializeObject in Json.Net?

Hello I'm using NewtonSoft Json.Net to deserialize my json data. I usually deserialize json string but I want to check all of not exist keys.
For example here is a json data.
{
"Hp": 100,
"PlayerInfo": {
"Atk": 10,
"Def": 20
},
"Mp": 100
}
And I have a structure which can match above data.
[Serializable]
public struct CharaData
{
public int Hp;
[Serializable]
public struct PlayerInfoData
{
public int Atk;
public int Def;
public int Spd;
}
PlayerInfoData PlayerInfo;
}
And I'm gonna deseialize it like this.
JsonConvert.DeserializeObject<CharaData>(jsonStr);
There is a Mp key in json data but in structure there is not.
And in PlayerInfoData there is no Spd key in json data but in structure there is a Spd field.
Well... Spd field seems initialize as a default 0 value and it could potentially be a bug.
So I want to check what keys are not in structure.
And what structure fields are not deserialized because of not exist.
I will do my best to prevent these happening, but if some keys are missing in the process of deserializing from json data, I will log to find the problem why deserialize wasn't
completely success.
[Error][CharaData::Mp key not exist in json string]
[Error][CharaData::PlayerInfo::Spd field not exist in struct]
Seems there is no any method to check it in JsonConvert class.
I saw
[JsonProperty(Required = Required.Always)]
but this does not check all of keys.
Is this need to write a custom json converter?

use this code
var result= JsonConvert.DeserializeObject<CharaData>(jsonStr);
var mp=result.Mp;
var playerInfo=result.PlayerInfo;
if you want to know what key are exist just check them for null. By default all keys are null. if they are not null, it means they took value from json. For example you can use this code
if (mp==null) Console.WriteLine ("mp is not exist in json");
another way is to use reflection to check all properties
var props = result.GetType().GetProperties();
var nulls = new List<string>();
foreach (var prop in props)
{
var propInstance = prop.GetValue(result, null);
if (propInstance == null) nulls.Add(prop.Name);
if (prop.Name == "PlayerInfo")
{
var prps = prop.PropertyType.GetProperties();
foreach (var prp in prps)
if (prp.GetValue(propInstance, null) == null) nulls.Add(prop.Name+"."+prp.Name);
}
}
foreach (var n in nulls)
Console.WriteLine(n + " doesn't have value");
test result
PlayerInfo.Spd doesn't have value
classes
public class PlayerInfo
{
public int? Atk { get; set; }
public int? Def { get; set; }
public int? Spd { get; set; }
}
public class CharaData
{
public int? Hp { get; set; }
public PlayerInfo PlayerInfo { get; set; }
public int? Mp { get; set; }
}

Your problem is twofold:
Find the missing fields
Find the extra fields
Before we are digging into the details let's split the CharaData into two classes
[Serializable]
public class CharaData
{
public int Hp;
public PlayerInfoData PlayerInfo;
}
[Serializable]
public class PlayerInfoData
{
public int Atk;
public int Def;
public int Spd;
}
Missing Fields
This solution relies on the JsonSchema
private static Lazy<JSchema> schema = new Lazy<JSchema>(() => {
var generator = new JSchemaGenerator();
return generator.Generate(typeof(CharaData));
}, true);
public static void ReportMissingFields(string json)
{
var semiParsed = JObject.Parse(json);
try
{
semiParsed.Validate(schema.Value);
}
catch (JSchemaValidationException ex)
{
Console.WriteLine(ex.ValidationError.Message);
}
}
schema stores the json schema of CharaData in a lazy fashion
Validate compares the json against the schema and if there is a mismatch then it throws a JSchemaValidationException
It exposes a property which type is ValidationError which contains a lots of information about the mismatch
Extra fields
This solution relies on JsonExtensionDataAttribute
[Serializable]
internal class CharaDataExtras: CharaData
{
[JsonExtensionData]
public IDictionary<string, JToken> ExtraFields;
}
...
public static void ReportExtraFields(string json)
{
var result = JsonConvert.DeserializeObject<CharaDataExtras>(json);
foreach (var field in result.ExtraFields)
{
Console.WriteLine($"An extra field has found, called {field.Key}");
}
}
I've defined the CharaData as class to be able to derive from it << CharaDataExtras
Every extra field will be put into the ExtraFields dictionary
Usage
var json = File.ReadAllText("sample.json");
ReportMissingFields(json);
ReportExtraFields(json);
The output:
Required properties are missing from object: Spd.
An extra field has found, called Mp

How can i see if my JSON contains a certain value and then compare it?

var Name = "Resources.myjson.json";
var NameJSON = new System.IO.StreamReader(typeof(Strings).GetTypeInfo().Assembly.GetManifestResourceStream(Name)).ReadToEnd();
var ParsedBrandJSON = Newtonsoft.Json.JsonConvert.DeserializeObject<TheInfo>(NameJSON);
await JsonCS.LoadJson(ParsedBrandJSON);
And on the page:
static public class TheInfoJSON
{
static public TheInfo Data { get; set; }
static public async Task LoadJson(Data JSON)
{
Data = JSON;
}
}
and
public class TheInfo
{
public List<TheName> TheName { get; set; } = new List<TheName>();
}
My json:
{
"TheInfo":[
{
"TheName": ["Martin", "Jonas", "Alex", "Oscar"]
}
]
}
When i now try to compare how can i see if my JSON contains a certain object and then store that as a single TheName? Is it possible to do it in the same cast?
var TheNameDataFromOtherPage = OtherPage.TheName; //Here i gather the name from another page that i will compare with the JSON
//Wrong syntax
bool DoTheyMatch = TheNameDataFromOtherPage == TheInfoJSON.Data.TheName.Contains("Alex");
This is now wrong syntax because i cant compare the value to a bool. How can i get out the data i find and then instead of having TheInfoJSON.Data.TheName.Contains("Alex"); as a bool, back to a single value of TheName containing "Alex" so I can create a bool out of the two values to see if the JSON has it or not.
I tried to add something along the lines like this after the contains(): as TheInfo.TheName but that isnt the correct syntax either.

bool DoTheyMatch = TheInfoJSON.Data.TheName.Contains(TheNameDataFromOtherPage);

Deserializing Json array with variable names first using C# Json.NET

I'm getting an irregular JSON array from the Census Bureau's public api.
The variable names are all in the first element, and I'm having trouble deserializing it.
http://api.census.gov/data/2014/pep/agesex?get=AGE,POP,SEX&for=us:*&DATE=7
gives me JSON like this:
[["AGE","POP","SEX","DATE","us"],
["0","3948350","0","7","1"],
["1","3962123","0","7","1"],
["2","3957772","0","7","1"],
["3","4005190","0","7","1"],
["4","4003448","0","7","1"],
["5","4004858","0","7","1"],
["6","4134352","0","7","1"],
["7","4154000","0","7","1"]]
I can successfully deserialize this using:
var test1 = JsonConvert.DeserializeObject<String[][]>(jsonStr);
However, I'm trying to deserialize it to a class like this:
public class TestClass
{
public string AGE { get; set; }
public string POP { get; set; }
public string SEX { get; set; }
public string DATE { get; set; }
public string us { get; set; }
}
I'm trying to do this:
var test2 = JsonConvert.DeserializeObject<TestClass[]>(jsonStr);
But I'm getting the following exception:
An exception of type 'Newtonsoft.Json.JsonSerializationException'
occurred in Newtonsoft.Json.dll but was not handled in user code
Additional information: Cannot create and populate list type
TestClass. Path '[0]', line 1, position
2.

There's two parts to this.
First is turning the JSON in to data usable in C#, and the second is turning that data in to nice objects.
Here's a working dotNetFiddle.net example of the following code: https://dotnetfiddle.net/Cr0aRL
Each row in your JSON is made up of an array of strings.
So that's an array of an array of strings.
In C# that can be written as string[][].
So to turn the JSON in to usable data with JSON.Net you can do:
var json = "[[\"AGE\",\"POP\",\"SEX\",\"DATE\",\"us\"],[\"0\",\"3948350\",\"0\",\"7\",\"1\"],[\"1\",\"3962123\",\"0\",\"7\",\"1\"],[\"2\",\"3957772\",\"0\",\"7\",\"1\"],[\"3\",\"4005190\",\"0\",\"7\",\"1\"],[\"4\",\"4003448\",\"0\",\"7\",\"1\"],[\"5\",\"4004858\",\"0\",\"7\",\"1\"],[\"6\",\"4134352\",\"0\",\"7\",\"1\"],[\"7\",\"4154000\",\"0\",\"7\",\"1\"]]";
var rawData = JsonConvert.DeserializeObject<string[][]>(json);
Next up is is turning that data in to objects.
The first row is the header, containing the column names, so we want to grab that, and then figure out the column index for each column name.
var headerRow = rawData.First();
var ageIndex = Array.IndexOf(headerRow, "AGE");
var popIndex = Array.IndexOf(headerRow, "POP");
var sexIndex = Array.IndexOf(headerRow, "SEX");
var dateIndex = Array.IndexOf(headerRow, "DATE");
var usIndex = Array.IndexOf(headerRow, "us");
Now we have the indexes, we need to take each row, and convert it in to the appropriate object. I've used LINQ for this as it's very good at representing data processing in a clear way.
var testData = rawData
.Skip(1) //The first row is a header, not data
.Select(dataRow => new TestClass()
{
AGE = dataRow[ageIndex],
POP = dataRow[popIndex],
SEX = dataRow[sexIndex],
DATE = dataRow[dateIndex],
us = dataRow[usIndex]
});
Finally a bit of testing, to make sure you have the data you're expecting.
//Get the second data row as an example
var example = testData.Skip(1).First();
//Output example POP to check value
Console.WriteLine(example.POP);
Everything above is very manual.
You have to know what headers you expect, then you manually find the indexes, then you manually map the rows to objects.
It's quite possible for a simple use case that doing that is fine. But in larger and/or more complex systems you might want/need to automate those steps.
Automating those steps is possible, but is beyond the scope of this answer as how you approach it can depend on a lot of different factors.

You could make a custom JsonConverter to handle this conversion during deserialization. The conversion code is really not much different than other answers here, except that it is encapsulated into a separate class so that you don't muddy up your main code with the conversion details. From the point of view of your main code it "just works".
Here is how to write the converter:
public class TestClassArrayConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(TestClass[]));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JArray table = JArray.Load(reader);
TestClass[] items = new TestClass[table.Count - 1];
for (int i = 1; i < table.Count; i++)
{
JArray row = (JArray)table[i];
items[i - 1] = new TestClass
{
AGE = (string)row[0],
POP = (string)row[1],
SEX = (string)row[2],
DATE = (string)row[3],
us = (string)row[4]
};
}
return items;
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
And here is how you would use it:
var test2 = JsonConvert.DeserializeObject<TestClass[]>(jsonStr, new TestClassArrayConverter());
Fiddle: https://dotnetfiddle.net/68Q0KT

You have to do the processing on your own, as there is no way the json deserializer can know, how to put the values into the respecitve variables.
If you know, this will be exactly this structure, you could for instance add an appropriate constructor
public TestClass(string[] values) {
AGE = values[0];
...
}
to your class. Then serialize your result to array of arrays of string and then pass the inner arrays to your constructor.
var t1 = JsonConvert.DeserializeObject<string[][]>(jsonStr);
//skip the first entry, as this contains the headers
var t2 = t1.Skip(1).Select(x=> new TestClass(x));
If your structure varies, you'll have to write some more complicated mapping code.

You will have to do some custom mapping as your Json does not have any naming conventions so you will have to work with the data in array and index formats. This will work:
var jsonStr = "[[\"AGE\",\"POP\",\"SEX\",\"DATE\",\"us\"], [\"0\",\"3948350\",\"0\",\"7\",\"1\"], [\"1\",\"3962123\",\"0\",\"7\",\"1\"], [\"2\",\"3957772\",\"0\",\"7\",\"1\"], [\"3\",\"4005190\",\"0\",\"7\",\"1\"], [\"4\",\"4003448\",\"0\",\"7\",\"1\"], [\"5\",\"4004858\",\"0\",\"7\",\"1\"], [\"6\",\"4134352\",\"0\",\"7\",\"1\"], [\"7\",\"4154000\",\"0\",\"7\",\"1\"]]";
var test2 = JsonConvert.DeserializeObject<string[][]>(jsonStr);
var test3 = test2.Select(x => new TestClass()
{
AGE = x[0].ToString(),
POP = x[1].ToString(),
SEX = x[2].ToString(),
DATE = x[3].ToString(),
us = x[4].ToString()
}).ToList();

//test Case
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
namespace ApiController.Test
{
[TestClass]
public class DownloadIrregularJsonStringObjects
{
string ApiKey => "YourPersonalCensusKey";
/// <summary>
/// You have to get your own ApiKey from the Census Website
/// </summary>
[TestMethod]
public void TestGetItem()
{
string url = $"http://api.census.gov/data/timeseries/healthins/sahie?get=NIC_PT,NAME,NUI_PT&for=county:*&in=state:*&time=2015&key={YourPersonalCensusKey}";
string expected = "Autauga County, AL";
IList<HealthData> actual = ApiController.DownloadIrregularJsonStringObjects.GetCensusHealthData(url);
Assert.AreEqual(actual[0].NAME, expected);
}
}
}
///Actual Assembly
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
namespace ApiController
{
public class DownloadIrregularJsonStringObjects
{
public static IList<HealthData> GetCensusHealthData(string url)
{
var json = GetData(url);
var rawData = JsonConvert.DeserializeObject<string[][]>(json);
var headerRow = rawData.First();
var nic_pt_Index = Array.IndexOf(headerRow, "NIC_PT");
var name_Index = Array.IndexOf(headerRow, "NAME");
var nui_pt_Index = Array.IndexOf(headerRow, "NUI_PT");
IList<HealthData> retVal = new List<HealthData>();
foreach (var r in rawData.Skip(1))
{
HealthData dataRow = new HealthData();
dataRow.NIC_PT = r[nic_pt_Index];
dataRow.NAME = r[name_Index];
dataRow.NUI_PT = r[nui_pt_Index];
retVal.Add(dataRow);
}
return retVal;
}
private static string GetData(string url)
{
using (var w = new WebClient())
{
var jsonData = string.Empty;
jsonData = w.DownloadString(url);
return jsonData;
}
}
}
public class HealthData
{
public string NIC_PT { get; set; }
public string NAME { get; set; }
public string NUI_PT { get; set; }
}
}

Ignore custom children with Json .Net

i have a json response like this:
{"response_values":[110,{"id":14753,"name":"piter"},{"id":14753,"name":"rabbit"}]}
and i have a simple class
public class Class1
{
[JsonProperty("id")]
public int Id { get; set; }
[JsonProperty("name")]
public string Name { get; set; }
}
and when i trying to convert json to list of the objects with this method:
public T Cast<T>(string json)
{
var result = default(T);
var jsonObject = JObject.Parse(json);
if (jsonObject != null)
{
var responseToken = jsonObject["response"];
result = responseToken.ToObject<T>();
}
return result;
}
like this
...
var jsonString = GetJson();
var items = Cast<List<Class1>>();
...
i have an exceiption, because value "110" is integer. How can i skip this value?

You always have this option if you expect the values to ignore to always be at the beginning:
if (jsonObject != null)
{
var responseToken = parsed["response_values"].SkipWhile(j => j.Type != JTokenType.Object);
if (responseToken.Count() > 0) result = responseToken.ToObject<T>();
}
You may prefer to use Skip(1) instead of SkipWhile if it's always the first value. Alternatively, you can use Where to ignore or select tokens anywhere in the message.
Of course, you can play around with this approach (changing things) depending on exactly what you expect to be returned in success scenarios.

print custom List in JavaScript

I have a custom list earthquakes which contains a list of earthquakes. How do I parse this in JavaScript in order to add it to innerHtml and display on the screen. The problem is that I cannot get this to display on the screen in a div. When it parses I get no result because my javascript is wrong and if I try just printing the result i get [Object object]
So the flow goes input from textbox -> web service -> list to javascript
earthquakes class:
public class earthquakes
{
public string eqid { get; set; }
public double magnitude { get; set; }
public double lng { get; set; }
public string source { get; set; }
public DateTime date { get; set; }
public int depth { get; set; }
public double lat { get; set; }
}
dataEarthquakes class
public class dataPostalCodes
{
public List<postalCodes> postalCodes { get; set; }
}
WebService:
public static dataEarthQuakes getEarthquakes(dataPostalCodes postalCodes)
{
double lat = postalCodes.postalCodes[0].lat;
double lng = postalCodes.postalCodes[0].lng;
Uri address = new Uri(String.Format(FindEarthquakes, lat, 0, lng, 0));
WebClient client = new WebClient();
string jsonResponse = string.Empty;
jsonResponse = client.DownloadString(address.AbsoluteUri);
var results = JsonConvert.DeserializeObject<dataEarthQuakes>(jsonResponse);
return results;
}
Javascript:
function OnLookupComplete(e) {
var result = e;
var weatherData = new Sys.StringBuilder();
var line;
for (var property in result.dataPostalCodes) {
line = String.format("<b>{0}:</b> {1}<br/>",
property, result.dataPostalCodes[property]);
weatherData.append(line);
}
$get('divResult').innerHTML = weatherData.toString();
}
Json string:
{"earthquakes":[{"eqid":"2010utc5","magnitude":7.7,"lng":97.1315,"src":"us","datetime":"2010-04-06 20:15:02","depth":31,"lat":2.3602}, {"eqid":"2009kdb2","magnitude":7.6,"lng":92.9226,"src":"us","datetime":"2009-08-10 17:55:39","depth":33.1,"lat":14.0129},{"eqid":"2010zbca","magnitude":7.6,"lng":123.533,"src":"us","datetime":"2010-07-23 20:51:11","depth":576.3,"lat":6.4939},{"eqid":"2010xkbv","magnitude":7.5,"lng":91.9379,"src":"us","datetime":"2010-06-12 17:26:50","depth":35,"lat":7.7477},{"eqid":"c0000rxc","magnitude":7.4,"lng":143.7392,"src":"us","datetime":"2010-12-21 16:19:41","depth":14.9,"lat":26.8656},{"eqid":"2010zbcd","magnitude":7.4,"lng":123.2677,"src":"us","datetime":"2010-07-23 21:15:08","depth":616.7,"lat":6.7489},{"eqid":"2010wbaq","magnitude":7.4,"lng":96.0805,"src":"us","datetime":"2010-05-09 03:59:44","depth":61.4,"lat":3.7284},{"eqid":"2007hvbq","magnitude":7.4,"lng":142.6846,"src":"us","datetime":"2007-09-28 11:38:58","depth":261.3,"lat":21.98},{"eqid":"2010zbbz","magnitude":7.3,"lng":123.4788,"src":"us","datetime":"2010-07-23 20:08:11","depth":604.5,"lat":6.7079},{"eqid":"2007xvam","magnitude":7.3,"lng":126.292,"src":"us","datetime":"2007-01-21 10:27:42","depth":10,"lat":1.2071}]}

As no.good.at.coding said in the comment, if your weatherData object contains the correct data, then it might be as simple as:
$('#divResult').html(weatherData.toString());
Another option may be to call parseJSON on your json object and then use jquery's each function to iterate through the results:
var results = $.parseJSON(e);
$(results).each(function (i, val) {
$('#divResult').append('<p>' + val.eqid + '<p>'); // can add markup here for magnitude and other properties
});
If you aren't sure what your objects are in javascript, firebug is a great tool for debugging (or you could use the Developer Tools that are built in to Chrome).
You didn't state an exact problem, but hopefully this will help get you on the right track.

Here's what I might do:
$.get('my-webservice-url',
function(data) {
OnLookupComplete(data['earthquakes']);
},
'json');
function OnLookupComplete(e) {
var weatherData = new Sys.StringBuilder();
for(var i=0;i<e.length;i++) {
var line;
for (var property in e[i].dataPostalCodes) {
line = String.format("<b>{0}:</b> {1}<br/>",
property, e[i].dataPostalCodes[property]);
weatherData.append(line);
}
}
$('#divResult').html(weatherData.toString());
}
The idea here is that you make your call to your web service and indicate to jQuery that the response expected is JSON (this is useful in case you aren't setting the content-type header correctly in the response from the server).
Once the GET request completes, jQuery will call your callback function (the anonymous function you can see in the call to $.get(). From your JSON example, I see that you expect an object earthquakes which is an array of objects of earthquake details.
The function then calls OnLookupComplete() with each the earthquakes array.
OnLookupComplete() then iterates over each earthquake, builds the right string and appends it to the StringBuilder. Finally, once all earthquakes have been dealt with, it appends the complete set of formatted lines to the div with the id divResult.

Categories