I'm getting an irregular JSON array from the Census Bureau's public api.
The variable names are all in the first element, and I'm having trouble deserializing it.
http://api.census.gov/data/2014/pep/agesex?get=AGE,POP,SEX&for=us:*&DATE=7
gives me JSON like this:
[["AGE","POP","SEX","DATE","us"],
["0","3948350","0","7","1"],
["1","3962123","0","7","1"],
["2","3957772","0","7","1"],
["3","4005190","0","7","1"],
["4","4003448","0","7","1"],
["5","4004858","0","7","1"],
["6","4134352","0","7","1"],
["7","4154000","0","7","1"]]
I can successfully deserialize this using:
var test1 = JsonConvert.DeserializeObject<String[][]>(jsonStr);
However, I'm trying to deserialize it to a class like this:
public class TestClass
{
public string AGE { get; set; }
public string POP { get; set; }
public string SEX { get; set; }
public string DATE { get; set; }
public string us { get; set; }
}
I'm trying to do this:
var test2 = JsonConvert.DeserializeObject<TestClass[]>(jsonStr);
But I'm getting the following exception:
An exception of type 'Newtonsoft.Json.JsonSerializationException'
occurred in Newtonsoft.Json.dll but was not handled in user code
Additional information: Cannot create and populate list type
TestClass. Path '[0]', line 1, position
2.
There's two parts to this.
First is turning the JSON in to data usable in C#, and the second is turning that data in to nice objects.
Here's a working dotNetFiddle.net example of the following code: https://dotnetfiddle.net/Cr0aRL
Each row in your JSON is made up of an array of strings.
So that's an array of an array of strings.
In C# that can be written as string[][].
So to turn the JSON in to usable data with JSON.Net you can do:
var json = "[[\"AGE\",\"POP\",\"SEX\",\"DATE\",\"us\"],[\"0\",\"3948350\",\"0\",\"7\",\"1\"],[\"1\",\"3962123\",\"0\",\"7\",\"1\"],[\"2\",\"3957772\",\"0\",\"7\",\"1\"],[\"3\",\"4005190\",\"0\",\"7\",\"1\"],[\"4\",\"4003448\",\"0\",\"7\",\"1\"],[\"5\",\"4004858\",\"0\",\"7\",\"1\"],[\"6\",\"4134352\",\"0\",\"7\",\"1\"],[\"7\",\"4154000\",\"0\",\"7\",\"1\"]]";
var rawData = JsonConvert.DeserializeObject<string[][]>(json);
Next up is is turning that data in to objects.
The first row is the header, containing the column names, so we want to grab that, and then figure out the column index for each column name.
var headerRow = rawData.First();
var ageIndex = Array.IndexOf(headerRow, "AGE");
var popIndex = Array.IndexOf(headerRow, "POP");
var sexIndex = Array.IndexOf(headerRow, "SEX");
var dateIndex = Array.IndexOf(headerRow, "DATE");
var usIndex = Array.IndexOf(headerRow, "us");
Now we have the indexes, we need to take each row, and convert it in to the appropriate object. I've used LINQ for this as it's very good at representing data processing in a clear way.
var testData = rawData
.Skip(1) //The first row is a header, not data
.Select(dataRow => new TestClass()
{
AGE = dataRow[ageIndex],
POP = dataRow[popIndex],
SEX = dataRow[sexIndex],
DATE = dataRow[dateIndex],
us = dataRow[usIndex]
});
Finally a bit of testing, to make sure you have the data you're expecting.
//Get the second data row as an example
var example = testData.Skip(1).First();
//Output example POP to check value
Console.WriteLine(example.POP);
Everything above is very manual.
You have to know what headers you expect, then you manually find the indexes, then you manually map the rows to objects.
It's quite possible for a simple use case that doing that is fine. But in larger and/or more complex systems you might want/need to automate those steps.
Automating those steps is possible, but is beyond the scope of this answer as how you approach it can depend on a lot of different factors.
You could make a custom JsonConverter to handle this conversion during deserialization. The conversion code is really not much different than other answers here, except that it is encapsulated into a separate class so that you don't muddy up your main code with the conversion details. From the point of view of your main code it "just works".
Here is how to write the converter:
public class TestClassArrayConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(TestClass[]));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JArray table = JArray.Load(reader);
TestClass[] items = new TestClass[table.Count - 1];
for (int i = 1; i < table.Count; i++)
{
JArray row = (JArray)table[i];
items[i - 1] = new TestClass
{
AGE = (string)row[0],
POP = (string)row[1],
SEX = (string)row[2],
DATE = (string)row[3],
us = (string)row[4]
};
}
return items;
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
And here is how you would use it:
var test2 = JsonConvert.DeserializeObject<TestClass[]>(jsonStr, new TestClassArrayConverter());
Fiddle: https://dotnetfiddle.net/68Q0KT
You have to do the processing on your own, as there is no way the json deserializer can know, how to put the values into the respecitve variables.
If you know, this will be exactly this structure, you could for instance add an appropriate constructor
public TestClass(string[] values) {
AGE = values[0];
...
}
to your class. Then serialize your result to array of arrays of string and then pass the inner arrays to your constructor.
var t1 = JsonConvert.DeserializeObject<string[][]>(jsonStr);
//skip the first entry, as this contains the headers
var t2 = t1.Skip(1).Select(x=> new TestClass(x));
If your structure varies, you'll have to write some more complicated mapping code.
You will have to do some custom mapping as your Json does not have any naming conventions so you will have to work with the data in array and index formats. This will work:
var jsonStr = "[[\"AGE\",\"POP\",\"SEX\",\"DATE\",\"us\"], [\"0\",\"3948350\",\"0\",\"7\",\"1\"], [\"1\",\"3962123\",\"0\",\"7\",\"1\"], [\"2\",\"3957772\",\"0\",\"7\",\"1\"], [\"3\",\"4005190\",\"0\",\"7\",\"1\"], [\"4\",\"4003448\",\"0\",\"7\",\"1\"], [\"5\",\"4004858\",\"0\",\"7\",\"1\"], [\"6\",\"4134352\",\"0\",\"7\",\"1\"], [\"7\",\"4154000\",\"0\",\"7\",\"1\"]]";
var test2 = JsonConvert.DeserializeObject<string[][]>(jsonStr);
var test3 = test2.Select(x => new TestClass()
{
AGE = x[0].ToString(),
POP = x[1].ToString(),
SEX = x[2].ToString(),
DATE = x[3].ToString(),
us = x[4].ToString()
}).ToList();
//test Case
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
namespace ApiController.Test
{
[TestClass]
public class DownloadIrregularJsonStringObjects
{
string ApiKey => "YourPersonalCensusKey";
/// <summary>
/// You have to get your own ApiKey from the Census Website
/// </summary>
[TestMethod]
public void TestGetItem()
{
string url = $"http://api.census.gov/data/timeseries/healthins/sahie?get=NIC_PT,NAME,NUI_PT&for=county:*&in=state:*&time=2015&key={YourPersonalCensusKey}";
string expected = "Autauga County, AL";
IList<HealthData> actual = ApiController.DownloadIrregularJsonStringObjects.GetCensusHealthData(url);
Assert.AreEqual(actual[0].NAME, expected);
}
}
}
///Actual Assembly
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
namespace ApiController
{
public class DownloadIrregularJsonStringObjects
{
public static IList<HealthData> GetCensusHealthData(string url)
{
var json = GetData(url);
var rawData = JsonConvert.DeserializeObject<string[][]>(json);
var headerRow = rawData.First();
var nic_pt_Index = Array.IndexOf(headerRow, "NIC_PT");
var name_Index = Array.IndexOf(headerRow, "NAME");
var nui_pt_Index = Array.IndexOf(headerRow, "NUI_PT");
IList<HealthData> retVal = new List<HealthData>();
foreach (var r in rawData.Skip(1))
{
HealthData dataRow = new HealthData();
dataRow.NIC_PT = r[nic_pt_Index];
dataRow.NAME = r[name_Index];
dataRow.NUI_PT = r[nui_pt_Index];
retVal.Add(dataRow);
}
return retVal;
}
private static string GetData(string url)
{
using (var w = new WebClient())
{
var jsonData = string.Empty;
jsonData = w.DownloadString(url);
return jsonData;
}
}
}
public class HealthData
{
public string NIC_PT { get; set; }
public string NAME { get; set; }
public string NUI_PT { get; set; }
}
}
Related
I am trying to serialize nested objects using reflection. I am able to do this fine for properties containing a single value, but I am having trouble with list type properties that contain another class.
In the code sample below I have a class Dish which contains a list of of Recipe classes as a property, which itself contains a list of Step classes.
I am able to get the PropertyInfo of the List property, but when I try to get the contents of it by invoking the get method, I get a simple object back, not a List of e.g. Steps:
var listObjects = property.GetGetMethod().Invoke(dish, null);
I managed to cast that into a List of objects like this:
List<object> listValues = ( listObjects as IEnumerable<object>).Cast<object>().ToList();
Now at least I can iterate over this List, but I cannot get the acutal properties of the original classes like the step description.
So I know the type of the List via property.PropertyType.GenericTypeArguments.First(), but its at runtime. I am thinking on how to perform a proper cast to transform my List<object> into a conrete type like List<Step>.
What I want to achieve: Serialize all property values of dish and all its attached Lists of objects.
I appreciate any ideas.
using System;
using System.Linq;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
var dish = new Dish(){
Recipes = new List<Recipe>(){
new Recipe(){
RecipeName = "Preparation",
Steps = new List<Step>(){
new Step(){
Description = "Prepare Stuff",
}
}
},
new Recipe(){
RecipeName = "Main Course",
Steps = new List<Step>(){
new Step(){
Description = "Do this",
},
new Step(){
Description = "Then do that",
}
}
}, }, };
var serializer = new Serializer();
serializer.SerializeDish(dish);
}
}
public class Serializer
{
public void SerializeDish(Dish dish)
{
var dishType = typeof (Dish);
var listProps = dishType.GetProperties().Where(x => (x.PropertyType.IsGenericType && x.PropertyType.GetGenericTypeDefinition() == typeof (List<>)));
foreach (var property in listProps)
{
var propertyGetMethod = property.GetGetMethod();
var listObjects = propertyGetMethod.Invoke(dish, null);
Console.WriteLine("Name:"+property.Name + " " + "List-Type:"+property.PropertyType.GenericTypeArguments.First());
//Here its getting fuzzy
List<object> listValues = ( listObjects as IEnumerable<object>).Cast<object>().ToList();
foreach ( var item in listValues ) {
Console.WriteLine(item);
}
}
}
}
public class Dish
{
public List<Recipe> Recipes {get;set;}
}
public class Recipe
{
public string RecipeName{get;set;}
public List<Step> Steps {get;set;}
}
public class Step
{
public string Description {get;set;}
}
I want to deserialize following JSON.
The problem is that the objects "ANDE" & "DAR" can change.
Means the objects are unknown and change depending on the JSON i wanna deserialize.
About 8000 different objects (ANDE, DAR, ...) need to be deserialized.
{"ANDE":
{"chart":[
{"date":"20180914","minute":"09:30"},{"date":"20180914","minute":"13:30"}]},
"DAR":
{"chart":[
{"date":"20180914","minute":"09:30"},{"date":"20180914","minute":"13:30"}]}}
I get the data by HTTP response and want to put into a List:
HttpResponseMessage response = client.GetAsync(API_PATH).GetAwaiter().GetResult();
List historicalDataList = response.Content.ReadAsAsync<List<HistoricalDataResponse>>().GetAwaiter().GetResult();
The HistoricalDataResponse class looks like:
public class HistoricalDataResponse
{
public string date { get; set; }
public string minute { get; set; }
}
How can i deserialize this kind of JSON with unknown objects in C#?
Any help is highly appreciated.
Then you should use a dynamic variable:
dynamic ReturnValue = JsonConvert.DeserializeObject(jsonstring);
note that as in dynamic objects, properties are determined after being assigned in runtime, so you will not get a drop down menu in design time, and also as its properties are unknown in design time, and property you test in design time even if its not correct, you wont get an error, and you will get the error in runtime when it is assigned.
dynamic ReturnValue = JsonConvert.DeserializeObject(jsonstring);
try
{
var a = ReturnValue.ANDE; // will work if it has ANDE property.
// do what you would do with ANDE
}
catch{}
try
{
var a = ReturnValue.DAR; // will work if it has DAR property.
// do what you would do with DAR
}
catch{}
Use a dictionary with string as key type :
void Main()
{
var client = new HttpClient();
HttpResponseMessage response = client.GetAsync("url").GetAwaiter().GetResult();
var json = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
var result = JsonConvert.DeserializeObject<Dictionary<string, DateResponse>>(json);
foreach (var element in result)
{
var key = element.Key; // ANDE
foreach (var item in element.Value.Chart)
{
var date = item.date;
var minute = item.minute;
}
}
}
public class DateResponse{
public List<HistoricalDataResponse> Chart { get; set; }
}
public class HistoricalDataResponse
{
public string date { get; set; }
public string minute { get; set; }
}
Consider the following mutable object:
class SomePoco
{
public int Id{get;set;}
public string Name{get;set;}
}
Let's round trip it through Json.NET:
var p=new SomePoco{Id=4,Name="spender"};
var json=JsonConvert.SerializeObject(p);
var pr = JsonConvert.DeserializeObject<SomePoco>(json);
Console.WriteLine($"Id:{pr.Id}, Name:{pr.Name}");
All is good.
Now, let's make out POCO immutable and feed values via a constructor:
class SomeImmutablePoco
{
public SomeImmutablePoco(int id, string name)
{
Id = id;
Name = name;
}
public int Id{get;}
public string Name{get;}
}
... and round-trip the data again:
var p = new SomeImmutablePoco(5, "spender's immutable friend");
var json = JsonConvert.SerializeObject(p);
var pr = JsonConvert.DeserializeObject<SomeImmutablePoco>(json);
Console.WriteLine($"Id:{pr.Id}, Name:{pr.Name}");
Still good.
Now, let's make a small change to our immutable class by renaming a constructor parameter:
class SomeImmutablePoco
{
public SomeImmutablePoco(int pocoId, string name)
{
Id = pocoId;
Name = name;
}
public int Id{get;}
public string Name{get;}
}
then:
var p = new SomeImmutablePoco(666, "diabolo");
var json = JsonConvert.SerializeObject(p);
var pr = JsonConvert.DeserializeObject<SomeImmutablePoco>(json);
Console.WriteLine($"Id:{pr.Id}, Name:{pr.Name}");
Oh dear... It looks like Json.NET is doing some reflective magic over the names of our constructor parameters and matching them to property names in our POCO/json. This means that our freshly deserialized object doesn't get an Id assigned to it. When we print out the Id, it's 0.
This is bad, and particularly troublesome to track down.
This problem might exist in a large collection of POCOs. How can I automate finding these problem POCO classes?
Here is the code that finds such classes using reflection:
var types = new List<Type>() { typeof(SomeImmutablePoco) }; // get all types using reflection
foreach (var type in types)
{
var props = type.GetProperties(bindingAttr: System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Instance);
foreach (var ctor in type.GetConstructors())
{
foreach (var param in ctor.GetParameters())
{
if (!props.Select(prop => prop.Name.ToLower()).Contains(param.Name.ToLower()))
{
Console.WriteLine($"The type {type.FullName} may have problems with Deserialization");
}
}
}
}
You can map your json property like that:
class SomeImmutablePoco
{
public SomeImmutablePoco(int pocoId, string name)
{
Id = pocoId;
Name = name;
}
[JsonProperty("pocoId")]
public int Id { get; }
public string Name { get; }
}
var Name = "Resources.myjson.json";
var NameJSON = new System.IO.StreamReader(typeof(Strings).GetTypeInfo().Assembly.GetManifestResourceStream(Name)).ReadToEnd();
var ParsedBrandJSON = Newtonsoft.Json.JsonConvert.DeserializeObject<TheInfo>(NameJSON);
await JsonCS.LoadJson(ParsedBrandJSON);
And on the page:
static public class TheInfoJSON
{
static public TheInfo Data { get; set; }
static public async Task LoadJson(Data JSON)
{
Data = JSON;
}
}
and
public class TheInfo
{
public List<TheName> TheName { get; set; } = new List<TheName>();
}
My json:
{
"TheInfo":[
{
"TheName": ["Martin", "Jonas", "Alex", "Oscar"]
}
]
}
When i now try to compare how can i see if my JSON contains a certain object and then store that as a single TheName? Is it possible to do it in the same cast?
var TheNameDataFromOtherPage = OtherPage.TheName; //Here i gather the name from another page that i will compare with the JSON
//Wrong syntax
bool DoTheyMatch = TheNameDataFromOtherPage == TheInfoJSON.Data.TheName.Contains("Alex");
This is now wrong syntax because i cant compare the value to a bool. How can i get out the data i find and then instead of having TheInfoJSON.Data.TheName.Contains("Alex"); as a bool, back to a single value of TheName containing "Alex" so I can create a bool out of the two values to see if the JSON has it or not.
I tried to add something along the lines like this after the contains(): as TheInfo.TheName but that isnt the correct syntax either.
bool DoTheyMatch = TheInfoJSON.Data.TheName.Contains(TheNameDataFromOtherPage);
I have a below Jason format in which the Claims will be a array collection now I need to read all the Claims(which is a collection in below json) related to the policyid from c# , could anyone help me resoling it
{
"ClaimList": [{
"Claims": ["CL8901"],
"PolicyID": "W1234sd",
"ClaimsCount": 1
}, {
"Claims": ["CL3456", "CL1234"],
"PolicyID": "T1234sd",
"ClaimsCount": 2
}, {
"Claims": ["CL1321"],
"PolicyID": "C1234sd",
"ClaimsCount": 1
}]
}
string json = Util.ReadLine<string>();
JObject.Parse(json).SelectToken("ClaimList").Select(cl => new
{
PolicyId = cl.SelectToken("PolicyID"),
Claims = cl.SelectToken("Claims").ToObject<List<string>>()
}).Dump();
Linqpad code
http://share.linqpad.net/vqoav5.linq
To achieve what you need, do the following:
Follow my answer here to create C# classes for your json
Use NewtonSoft NuGet library to deserialize the JSON to C# objects
Perform whatever processing you need on the deserialized items
Here is some code once you have done the above:
var claims = JsonConvert.DeserializeObject<Rootobject>("YourJsonHere");
IEnumerable<Claimlist> claimsForPolicy=
claims.ClaimList.Where(x => x.PolicyID == "whateverYouNeed");
If you can use the Json.NET library (NuGet), this is pretty straightforward. First make sure you have a Claim class that has the structure of those inner objects in your JSON:
class Claim
{
public List<string> Claims { get; set; }
public string PolicyID { get; set; }
public int ClaimsCount { get; set; }
}
Having that class available, you can use Json.NET to do something like this:
using Newtonsoft.Json;
// ...
string yourJson = getYourJsonStringSomehow();
var claimsDict =
JsonConvert.DeserializeObject<Dictionary<string, List<Claim>>>(yourJson);
That will produce a Dictionary with one key: "ClaimsList", and the value will be a List of your Claim objects:
var claimsList = claimsDict["ClaimsList"];
var firstClaim = claimsList[0];
Console.WriteLine(firstClaim.PolicyID);
// Should print W1234sd
You can try this, first lets create a Json class for your json data:
public class JsonDoc
{
//I am adding an index on constructor for looping purposes
public JsonDoc(string json, int index)
{
JObject jObject = JObject.Parse(json);
JToken jResult = jObject["ClaimList"];
Claims = jResult[index]["Claims"].ToObject<string[]>();
}
//This constructor will return the number of count under ClaimList node
public JsonDoc(string json)
{
JObject jObject = JObject.Parse(json);
JToken jResult = jObject["ClaimList"][0];
intCount = jResult.Count();
}
private string[] Claims { get; set; }
public int intCount { get; set; }
}
Then to use this class for your Json data, first add a reference of Newtonsoft Json.
using Newtonsoft.Json.Linq;
Then on your sample data, I read all the data via file using StreamReader, loop on it and put all the data under a List:
private static void Main(string[] args)
{
using (StreamReader sr = new StreamReader(#"C:\Folder\myjson.json"))
{
string json = sr.ReadToEnd();
JsonDoc jSon = new JsonDoc(json);
//This will get the length of the array under ClaimList node
int intCount = jSon.intCount;
List<JsonDoc> lstResult = new List<JsonDoc>();
for (int x = 0; x < intCount; x++)
{
lstResult.Add(new JsonDoc(json, x));
}
}
}
The result is a List of JsonDoc class with array of string as part of property. This array of string is the result on your Claims Node. This JsonDoc class contains property Claims which is a string array which contains your Claims node: