I need to convert JSON data that I get from a REST API and convert them to CSV for some analytic. The problem is that the JSON data do not necessarily follow the same content, so I can't define a type for mapping. This has become a challenge that is taking too much of my time. I have already created some code, but of course it is not working as it throws exception on this line
var data = JsonConvert.DeserializeObject<List<object>>(jsonData);
The error is:
Additional information: Cannot deserialize the current JSON object
(e.g. {"name":"value"}) into type
'System.Collections.Generic.List`1[System.Object]' because the type
requires a JSON array (e.g. [1,2,3]) to deserialize correctly.
To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.
Path 'data', line 2, position 10.
please let me know what I can do to get this going.
A sample of data would be like this, the fields of data can change very often, for example a new field can be added the next day, so I don't have the liberty to create a .Net class to map the data.
{
"data": [
{
"ID": "5367ab140026875f70677ab277501bfa",
"name": "Happiness Initiatives - Flow of Communication/Process & Efficiency",
"objCode": "PROJ",
"percentComplete": 100.0,
"plannedCompletionDate": "2014-08-22T17:00:00:000-0400",
"plannedStartDate": "2014-05-05T09:00:00:000-0400",
"priority": 1,
"projectedCompletionDate": "2014-12-05T08:10:21:555-0500",
"status": "CPL"
},
{
"ID": "555f452900c8b845238716dd033cf71b",
"name": "UX Personalization Think Tank and Product Strategy",
"objCode": "PROJ",
"percentComplete": 0.0,
"plannedCompletionDate": "2015-12-01T09:00:00:000-0500",
"plannedStartDate": "2015-05-22T09:00:00:000-0400",
"priority": 1,
"projectedCompletionDate": "2016-01-04T09:00:00:000-0500",
"status": "APR"
},
{
"ID": "528b92020051ab208aef09a4740b1fe9",
"name": "SCL Health System - full Sitecore implementation (Task groups with SOW totals in Planned hours - do not bill time here)",
"objCode": "PROJ",
"percentComplete": 100.0,
"plannedCompletionDate": "2016-04-08T17:00:00:000-0400",
"plannedStartDate": "2013-11-04T09:00:00:000-0500",
"priority": 1,
"projectedCompletionDate": "2013-12-12T22:30:00:000-0500",
"status": "CPL"
}
]
}
namespace BusinessLogic
{
public class JsonToCsv
{
public string ToCsv(string jsonData, string datasetName)
{
var data = JsonConvert.DeserializeObject<List<object>>(jsonData);
DataTable table = ToDataTable(data);
StringBuilder result = new StringBuilder();
for (int i = 0; i < table.Columns.Count; i++)
{
result.Append(table.Columns[i].ColumnName);
result.Append(i == table.Columns.Count - 1 ? "\n" : ",");
}
foreach (DataRow row in table.Rows)
{
for (int i = 0; i < table.Columns.Count; i++)
{
result.Append(row[i].ToString());
result.Append(i == table.Columns.Count - 1 ? "\n" : ",");
}
}
return result.ToString().TrimEnd(new char[] {'\r', '\n'});
}
private DataTable ToDataTable<T>( IList<T> data )
{
PropertyDescriptorCollection props = TypeDescriptor.GetProperties(typeof(T));
DataTable table = new DataTable();
for (int i = 0 ; i < props.Count ; i++)
{
PropertyDescriptor prop = props[i];
table.Columns.Add(prop.Name, prop.PropertyType);
}
object[] values = new object[props.Count];
foreach (T item in data)
{
for (int i = 0 ; i < values.Length ; i++)
{
values[i] = props[i].GetValue(item);
}
table.Rows.Add(values);
}
return table;
}
}
}
The real issue here is that you are trying to deserialize into a List<object> but your JSON actually represents a single object containing a data property which then contains a list of objects. That is why you are getting this error. Json.Net can't deserialize a single object into a list. I think what you really want to do is define a container class like this:
class Root
{
public List<Dictionary<string, object>> Data { get; set;}
}
Then deserialize like this:
var data = JsonConvert.DeserializeObject<Root>(jsonData).Data;
You will then end up with a list of dictionaries, where each dictionary represents one item in the JSON array. The dictionary key-value pairs are the dynamic values in each item. You can then work with these as you would with any other dictionary. For example, here is how you would dump out all the data:
foreach (var dict in data)
{
foreach (var kvp in dict)
{
Console.WriteLine(kvp.Key + ": " + kvp.Value);
}
Console.WriteLine();
}
Fiddle: https://dotnetfiddle.net/6UaKhJ
What you're looking for is the dynamic type. Though unrelated, this answer contains much of the information on how you'll be able to iterate through the changing properties on your object.
You will need to add some additional work to figure out how to handle your result when it is an array versus a single object as your error shows us. However, this is a good first step for you.
Basically, a dynamic object is a Dictionary, much like how a JSON object is treated in JavaScript. You just need to iterate through each of the KeyValuePair objects within the main object and go through their properties.
var data = JsonConvert.DeserializeObject<dynamic>(jsonData);
var rows = new List<string>();
// Go through the overall object, and get each item in
// the array, or property in a single object.
foreach (KeyValuePair<string, object> item in data)
{
dynamic obj = item.Value;
var row = "";
// Perhaps add a check here to see if there are more
// properties (if it is an item in an array). If not
// then you are working with a single object, and each
// item is a property itself.
foreach (KeyValuePair<string, object> prop in obj)
{
// Very dummy way to demo adding to a CSV
string += prop.Value.ToString() + ",";
}
rows.Add(string);
}
This is far from a complete example, but we don't have enough information to go on to help you finish what you're trying to do.
You are trying to deserialize into a List but your JSON actually represents a single object containing a data property containing list of objects. That is why you are getting this error. Json.Net can't deserialize a single object into a list.
Please try this:Create a class which contain single property on datatype Object and pass this class for deserialization.
class Parent
{
public object Data { get; set;}
}
Then deserialize like this:
var output = JsonConvert.DeserializeObject<Parent>(jsonData);
Try using this class instead of Object
public class Datum
{
public string ID { get; set; }
public string name { get; set; }
public string objCode { get; set; }
public double percentComplete { get; set; }
public string plannedCompletionDate { get; set; }
public string plannedStartDate { get; set; }
public int priority { get; set; }
public string projectedCompletionDate { get; set; }
public string status { get; set; }
}
public class RootObject
{
public List<Datum> data { get; set; }
}
Change to this:
var data = JsonConvert.DeserializeObject<RootObject>(jsonData);
If your data is dynamic so try a dynamic list:
using System.Web.Script.Serialization;
JavaScriptSerializer jss = new JavaScriptSerializer();
var d=jss.Deserialize<dynamic>(str);
Since you're trying to deserialize an object type into a list type, it won't deseralize directly.
You can do this:
var data = JsonConvert.DeserializeObject<ObjectDataList>(jsonData);
var rows = new List<DeserializedData>();
foreach (dynamic item in data)
{
var newData = new DeserializedData();
foreach (dynamic prop in item)
{
var row = new KeyValuePair<string, string>
(prop.Name.ToString(), prop.Value.ToString());
newData.Add(row);
}
rows.Add(newData);
}
Here are new classes
//class for key value type data
class DeserializedData
{
List<KeyValuePair<string, string>> NewData =
new List<KeyValuePair<string, string>>();
internal void Add(KeyValuePair<string, string> row)
{
NewData.Add(row);
}
}
[DataContract]
class ObjectDataList
{
[DataMember(Name ="data")]
List<object> Data { get; set; }
public IEnumerator<object> GetEnumerator()
{
foreach (var d in Data)
{
yield return d;
}
}
}
As far as I can tell, more recent versions of Newtonsoft can actually do this now, no additional work required.
I was working with the binary version however, and this did still have the issue - I had a test where you could configure to use binary or json, and the json version worked just fine, but the binary complained about not getting an array type.
I started using the BsonDataReader for this, and was looking through the properties and methods on it to see how I could best look at the contents, and lo and behold, this had a property called:
reader.ReadRootValueAsArray
Setting this to 'true' did the trick.
Related
I am trying to deserialize a json but when the array ("groups") is empty it gives me an error. How can I check in Unity that if there is no data in the "groups" field a message will appear, for example?
This is the error:
InvalidCastException: Specified cast is not valid.
BC_Equipos.successCallbackGMGI (System.String jsonResponse, System.Object cbObject) (at Assets/_AG/Scripts/BC_Equipos.cs:249)
The error line 249 is: var groups = (Dictionary<string, object>[])jsonData["groups"];
When there is data in "groups" all works fine.
The json is:
{
"data": {
"requested": [],
"invited": [],
"groups": []
},
"status": 200
}
Here is how I'm deserializating:
var jsonMessage = (Dictionary<string, object>)BrainCloud.JsonFx.Json.JsonReader.Deserialize(jsonResponse);
var jsonData = (Dictionary<string, object>)jsonMessage["data"];
var groups = (Dictionary<string, object>[])jsonData["groups"];
if (groups.Length > 0)
{
Debug.Log("Groups List");
foreach (var group in groups)
{
var groupId = group["groupId"];
Debug.Log("groupId: " + groupId);
var nameGroup = group["name"];
Debug.Log("nameGroup: " + nameGroup);
}
}
else
{
Debug.Log("User hasn't joined a group already!");
}
If the array is empty, the tool can't infer what kind of array to create - typical data in the general case could, after all, be ["abc", "def"], [12, 42, 42], or - as seems to be the case - [{ }, { }]. It could even be something hybrid like ["abc", 42, {}]. You may have better luck just casting to Array - or perhaps object[], checking the length, and going from there. If object[] doesn't work: try Array, IList, or similar.
Or: a much better approach is to create a type model, and deserialize into that; then the serializer can use the defined model to interpret the data. For example:
public class SomeRoot {
public SomeData data {get;set;}
public int status {get;set;}
}
public class SomeData {
public List<Request> requested {get;} = new();
public List<Invite> invited {get;} = new();
public List<Group> groups {get;} = new();
}
public class Request {}
public class Invite {}
public class Group {}
I am trying to get value of label from following string.
I have a string in this format.
var string = "[{\"key\":\"182\",\"label\":\"testinstitution\"}]"
dynamic dict = new JavaScriptSerializer().Deserialize<dynamic>(string);
string inst = dict["label"]; //This is not working
I am getting dict in form of key value pair object but I am not able to get value of label. I cannot use JSON.NET.
To retrieve value of label from your string use string inst = dict[0]["label"];.
Explanation
The reason why you need additional [0] is because deserialization returns array of key value pairs. First object from that array will go to index [0], second object from array to index [1] an so on. Your string has array of only one object. Here is an example of when you have two objects, where second object has another object inside of it, in which case you would have to write dict[1]["foo"]["two"] to get to desired value:
var myString = #"
[
{
'one': '1'
},
{
'foo':
{
'two': '2'
}
}
]";
dynamic dict = new JavaScriptSerializer().Deserialize<dynamic>(myString);
string inst = dict[1]["foo"]["two"];
Additional FYI
If you know structure of your data consider using strong types (as suggested in one of the comments). Here is example of how you would do it:
public class Data
{
public string key { get; set; }
public string label { get; set; }
}
class Program
{
static void Main(string[] args)
{
var myString = #"
[{
'key': 182,
'label': 'testinstitution'
}]";
List<Data> dict = new JavaScriptSerializer().Deserialize<List<Data>>(myString);
foreach (var d in dict)
Console.WriteLine(d.key + " " + d.label);
Console.ReadKey();
}
}
Note that properties key and value in your Data object much match names of those in your array of objects exactly.
I am working on census data and the url for my json is:
http://api.census.gov/data.json
From the above json i want to build the drop down of "title" where "c_dataset" have "acs5". Now the code in my MVC controller is as follows :
var yourObject = new JavaScriptSerializer().Deserialize<dynamic>(result.ToString());
foreach (KeyValuePair<string, object> currency in yourObject)
{
if (currency.Key == "dataset")
{
foreach (KeyValuePair<string, object> cur in currency.Value)
{
}
}
}
the inner most foreach gives me an error that "foreach statement can not operate on variable type 'object' because 'object' does not contain a public definition for 'GetEnumerator' !! i can not deserialize the object in certain class because this json is big. What could be the best way to do this ? Also, i was wondering if i can use the LINQ to query the deserialize object. Thanks in advance.
I think your best bet would be to build concrete type(s) that only have the properties you want, ignoring everything else. JSON deserializer will then ignore additional "noise" and only read the properties you specified.
// not tested
class CensusDataSet
{
[JsonProperty("c_dataset")]
public string[] CDataset { get; set; }
[JsonProperty("title")]
public string Title { get; set; }
}
class CensusData
{
[JsonProperty("dataset")]
public CensusDataSet[] DataSets { get; set; }
}
...
var data = Newtonsoft.Json.JsonConvert.DeserializeObject<CensusData>(content);
foreach (var dataSet in data.DataSets) ...
BTW, your title is misleading: what you are trying to do is called deserialize.
This is what my code looks like in my public Form1()
while (accessReader.Read())
{
for (int i = 0; i < count; i++)
{
string urlpart2= accessReader.GetValue(i).ToString();
WebRequest request = WebRequest.Create("urlpart1" + urlpart2+ "urlpart3");
string json;
var response = request.GetResponse();
request.ContentType = "application/json; charset=utf-8";
using (var streamr = new StreamReader(response.GetResponseStream()))
{
json = streamr.ReadToEnd();
List<MyObject> list = JsonConvert.DeserializeObject<List<MyObject>>(json);
var date = MyObject.Start;
//MessageBox.Show(date.ToString());
This is my class representing the different variables my json string returns
public class MyObject
{
public int Type { get; set; }
public string Country { get; set; }
public string Channel { get; set; }
public string Code { get; set; }
public DateTime Start { get; set; }
public DateTime End { get; set; }
}
This is what a json string will return, the only difference that will change is maybe type, code, start, and end. - I want the Start and End Values.
[{"Type":1,"Country":"CA","Channel":"","Code":"1EZ","Start":"2014-10-24T00:00:00","End":"2015-10-23T00:00:00"},{"Type":2,"Country":"","Channel":"","Code":"UAD","Start":"2014-10-24T00:00:00","End":"2017-10-23T00:00:00"},{"Type":2,"Country":"","Channel":"","Code":"TPQ","Start":"2014-10-24T00:00:00","End":"2017-10-23T00:00:00"},{"Type":3,"Country":"","Channel":"","Code":"SVC_PRIORITY","Start":"2014-10-24T00:00:00","End":"2017-10-23T00:00:00"}]
I am fairly new to programming and I have run into an error I do not really understand 'An object reference is required for the nonstatic field, method, or property'.
Also I created this class within my Form1.cs as opposed to creating a new class in my Project Solution (if that matters?)
You try to access MyObject.Start like it would be a static property or field. However it's a instance field and basically you need to get a instance to access the field.
MyObject obj = list[0];
var date = obj.Start;
The field List<MyObject> list = ... contains all data which got deserialized from the json file ( And none if no data exists in the json file! ).
I guess you want to access all data of the json file, in this case i would prefer iterating through the objects via a foreach loop.
foreach (MyObject item in list) { .... }
The problem is with line var date = MyObject.Start; You have created a list of MyObjects. To access them you have to use list indexes i.e. var first = list[0] returns first element. Then you can get required date as: first.Start
In order to process every entry of your list you can use foreach loop as follows:
foreach (var obj in list) {
// use obj.Start, obj.End values
}
I have an incoming JSON, which consists array of some objects, say, Foo. I deserialize them with
result = JsonConvert.DeserializeObject<List<Foo>>(message);
Now i want to add a string property to Foo, which will store it's JSON (which i received), so that Foo'll look like:
public class Foo
{
public int MyInt { get; set; }
public bool MyBool { get; set; }
public string JSON { get; set; }
}
But i don't know how can i say JSON.Net the way it can populate such a field..
UPD
I'll clearify what i want. Say i receive JSON:
[{"MyInt":1,"MyBool":0},{"MyInt":2,"MyBool":0},{"MyInt":3,"MyBool":1}]
Here is array of 3 objects and i want, when deserializing, to add corresponding part of json to object, so that:
First object will contain {"MyInt":1,"MyBool":0}
Second object will contain {"MyInt":2,"MyBool":0}
Third object will contain {"MyInt":3,"MyBool":1}
in their JSON Property
I'll be gratefull for any help!
This is one way to do it, but it doesn't maintain the exact original JSON - but it does provide a static record of the original JSON (but without the exact format of the original values - i.e. Bool maybe be 0/1 or true/false):
var message = #"[{""MyInt"":1,""MyBool"":0},{""MyInt"":2,""MyBool"":0},{""MyInt"":3,""MyBool"":1}]";
var foos = JsonConvert.DeserializeObject<List<Foo>>(message);
var t = JsonConvert.SerializeObject(foos[0]);
foos = foos.Select(s => new Foo() { MyBool = s.MyBool, MyInt = s.MyInt, JSON = JsonConvert.SerializeObject(s) }).ToList();
If you are dealing with a lot of Foos, then you might want to find a more efficient way. There might be a way to 'update' using linq, rather than creating a new list.
Okay, i found an answer. I didn't know that i can deserialize message into JArray and then enumerate it (good job, newtonsoft:) ). Here is what i endede up with:
if (tokenType is JArray)
{
var arr = JsonConvert.DeserializeObject(message) as JArray;
foreach (var item in arr)
{
try
{
var agentParameter = item.ToObject<Foo>();
agentParameter.JSON = item.ToString();
result.Add(agentParameter);
}
catch (Exception)
{
LogProvider.Error(string.Format("Failed to Deserialize message. Message text: \r\n {0}", item.ToString()));
}
}
}