We Keep Coding

Convert Nested JSON to CSV in C# via ChoETL

Does anyone know how to convert the below nested JSON to CSV via CHOETL (An ETL framework for .NET)? Thank you!
I'm using this code but it will only return the first equipment record.
CODE:
{
using (var json = new ChoJSONReader("./test.json"))
{
csv.Write(json.Cast<dynamic>().Select(i => new
{
EquipmentId = i.GpsLocation.Equipment[0].EquipmentId,
InquiryValue = i.GpsLocation.Equipment[0].InquiryValue,
Timestamp = i.GpsLocation.Equipment[0].Timestamp
}));
}
}
JSON:
"GpsLocation": {
"Equipment": [
{
"EquipmentId": "EQ00001",
"InquiryValue": [
"IV00001"
],
"Timestamp": "2020-01-01 01:01:01.01",
},
{
"EquipmentId": "EQ00002",
"InquiryValue": [
"IV00002"
],
"Timestamp": "2020-01-01 01:01:01.01"
}
]
}
}````

As others suggest, the issue is you are only looking at the first element of the array.
It appears that the easiest way to control what you serialise into CSV is by correctly defining your source objects from JSON. JSON Path expressions come in pretty handy.
What I ended up doing here is query all JSON to return an array of Equipment objects regardless of where they are in the hierarchy (which means you may need to filter it a bit better depending on your full JSON).
Then it's pretty easy to define each field based on JSON path and just pass the result to CSVWriter.
Also check out some gotchas that I outlined in the respective comment lines.
void Main()
{
var jsonString = "{\"GpsLocation\":{\"Equipment\":[{\"EquipmentId\":\"EQ00001\",\"InquiryValue\":[\"IV00001\"],\"Timestamp\":\"2020-01-01 01:01:01.01\"},{\"EquipmentId\":\"EQ00002\",\"InquiryValue\":[\"IV00002\"],\"Timestamp\":\"2020-01-01 01:01:01.01\"}]}}";
var jsonReader = new StringReader(jsonString);
var csvWriter = new StringWriter(); // outputs to string, comment out if you want file output
//var csvWriter = new StreamWriter(".\\your_output.csv"); // writes to a file of your choice
using (var csv = new ChoCSVWriter(csvWriter))
using (var json = new ChoJSONReader(jsonReader)
.WithJSONPath("$..Equipment[*]", true) // firstly you scope the reader to all Equipment objects. take note of the second parameter. Apparently you need to pass true here as otherwise it just won't return anythig
.WithField("EquipmentId", jsonPath: "$.EquipmentId", isArray: false) // then you scope each field in the array to what you want it to be. Since you want scalar values, pass `isArray: false` for better predictability
.WithField("InquiryValue", jsonPath: "$.InquiryValue[0]", isArray: false) // since your InquiryValue is actually an array, you want to obtain first element here. if you don't do this, fields names and values would go askew
.WithField("Timestamp", jsonPath: "$.Timestamp", fieldType: typeof(DateTime), isArray: false)) // you can also supply field type, otherwise it seems to default to `string`
{
csv.WithFirstLineHeader().Write(json);
}
Console.WriteLine(csvWriter.GetStringBuilder().ToString()); // comment this out if writing to file - you won't need it
}
Update summary:
Pivoted to update the code to rely on JSON Path scoping - this seems to allow for field name manipulation with pretty low effort
Looking at your comment, you could probably simplify your file writer a little bit - use StreamWriter instead of StringWriter - see updated code for example

Here is the working sample of producing CSV from your JSON
string json = #"{
""GpsLocation"": {
""Equipment"": [
{
""EquipmentId"": ""EQ00001"",
""InquiryValue"": [
""IV00001""
],
""Timestamp"": ""2020-02-01 01:01:01.01"",
},
{
""EquipmentId"": ""EQ00002"",
""InquiryValue"": [
""IV00002""
],
""Timestamp"": ""2020-01-01 01:01:01.01""
}
]
}
}";
StringBuilder csv = new StringBuilder();
using (var r = ChoJSONReader.LoadText(json)
.WithJSONPath("$.GpsLocation.Equipment")
.WithField("EquipmentId")
.WithField("InquiryValue", jsonPath: "InquiryValue[0]", fieldType: typeof(string))
.WithField("Timestamp", fieldType: typeof(DateTime))
)
{
using (var w = new ChoCSVWriter(csv)
.WithFirstLineHeader())
w.Write(r);
}
Console.WriteLine(csv.ToString());
Output:
EquipmentId,InquiryValue,Timestamp
EQ00001,IV00001,2/1/2020 1:01:01 AM
EQ00002,IV00002,1/1/2020 1:01:01 AM
Sample fiddle: https://dotnetfiddle.net/hJWtqH

Your code is sound, but the issue is that you're only writing the first variable in the array by using i.GpsLocation.Equipment[0]. Instead, try looping over everything by putting it into a for loop, and changing the [0] to your iterating variable inside of said loop.

Error when trying to extract values from a Json Object C#

I have a Json string like below and this is only a small snippet. The number in quotation marks is a Unix Time which i will need to use to iterate over each object.
{
"result": {
"1534860000": [
"1534860000",
19,
41
],
"1534863600": [
"1534863600",
11,
16
],
"1534867200": [
"1534867200",
2,
5
]
}
}
But when I attempt to extract the data in the arrays I get an error:
System.InvalidOperationException: 'Cannot access child value on Newtonsoft.Json.Linq.JProperty.'
Code:
JObject jsonObj = JObject.Parse(response);
string unixTime = Helpers.ConvertToUnix(yesterday.AddHours(hour)).ToString();
foreach (var obj in jsonObj["result"])
{
var array = obj[unixTime]; //here is where the error occurs
}
Anyone able to shed some light on what I am missing?

If we simplify your example code a little to remove the unixTime element (let's just hardcode it for now), we end up with this:
JObject jsonObj = JObject.Parse(response);
string unixTime = "1534860000";
At this stage, we have jsonObj which refers to the root of the JSON object and has a single property of result. Repeating your foreach here for context:
foreach (var obj in jsonObj["result"])
{
var array = obj[unixTime]; //here is where the error occurs
}
You end up with obj referring to the JSON path of result.1534860000. The problem is you're then looking for a property 1534860000 at this JSON path (result.1534860000.1534860000), which does not exist.
You can just get the value directly, like so:
var array = obj["result"][unixTime]
Of course, this requires some error-checking for ensuring the path exists, etc, but it demonstrates the point.

After some help from Kirk Larkin I thought I would post a code snippet up.
JObject jsonObj = JObject.Parse(response);
int hour = 0;
string unixTime = Helpers.ConvertToUnix(yesterday.AddHours(hour)).ToString();
var array = jsonObj["result"][unixTime];
It now returns the contents of the array.

C# - JObject.Parse - Invalid JSON

I'm working with an API that is returning JSON.
I have a method that calls the api, and parses the JSON response for the desired nodes.
Up to this point everything has been working fine, except the latest JSON response appears to be malformed.
Other responses come back like:
{
"Keyword":"\"marhope\"",
"TermKey":null,
"Customers":[
{
"Memberships":[ ],
"CompanyId":0,
"ObjectId":112974,
"ObjectType":"Customer",
}
]
}
I use JObject.Parse to bring back the appropriate nodes by name.
The latest JSON response comes back as:
{
[
{
"AnimalId":9079117,
"SpeciesCode":"XX",
}
]
}
As you can see, there is no "name", and the JSON is slightly invalid.
How can I parse this. For the first example I was using the code below, but now that the JSON has no "name", I don't know how to approach this, thoughts?
JObject results = JObject.Parse(csr.SearchCustomer(1, 1, 870, term));
foreach (var resp in results["Customers"])
{
string obj = (string)resp["CompanyId"];
}

Jon Skeet is correct, the second JSON is invalid: you cannot have an array directly inside an object with no property name. The best course of action is to get the API developers to fix the JSON. However, if you're just looking for a quick and dirty workaround, you could strip off the the first and last brace from the invalid JSON and then parse it as an array using JArray.Parse.
string json = #"{
[
{
""AnimalId"":9079117,
""SpeciesCode"":""XX"",
}
]
}";
json = json.Substring(1, json.Length - 2);
JArray array = JArray.Parse(json);
foreach (JObject item in array.Children<JObject>())
{
Console.WriteLine("AnimalId: " + item["AnimalId"]);
Console.WriteLine("SpeciesCode: " + item["SpeciesCode"]);
}

converting graph api for xml

I'm having trouble converting a string of json facebook graph api, I used the facebook C# and json.Net.
But at conversion time it returns this error: Name can not begin with the '0 'character, hexadecimal value 0x30.
This is the code:
dynamic result = await _fb.GetTaskAsync ("me / feed");
FBxml JsonConvert.DeserializeXNode string = (result.ToString ()). ToString ();

It looks like there is a problem with portion of the json string as mentioned below (taken from your link http://jsfiddle.net/btripoloni/PaLC2/)
"story_tags": {
"0": [{
"id": "100000866891334",
"name": "Bruno Tripoloni",
"offset": 0,
"length": 15,
"type": "user"}]
},
Json cannot create class that begins with a numeric value such as '0'. Try creating the classes using the link http://json2csharp.com/ you will get an idea.
To solve this problem you can create a dynamic object and go through each properties OR create a JsonConverter and write your code in the ReadJson to convert the "0" to a meaningful name. May be this can help you http://blog.maskalik.com/asp-net/json-net-implement-custom-serialization
If this is not your problem then update the question with more information like class structure of FBxml, call stack of the exception (from which line of the json code is throwing the exception), Json version etc.

As keyr says, the problem is with those JSON properties that have numeric names. In XML names can contain numeric characters but cannot begin with one: XML (see the Well-formedness and error-handling section).
My idea was to recursively parse the JSON with JSON.Net, replacing properties that had numeric names:
var jsonObject = JObject.Parse(json);
foreach (var obj in jsonObject)
{
Process(obj.Value);
}
XDocument document = JsonConvert.DeserializeXNode(jsonObject.ToString());
....
private static void Process(JToken jToken)
{
if (jToken.Type == JTokenType.Property)
{
JProperty property = jToken as JProperty;
int value;
if (int.TryParse(property.Name, out value))
{
JToken token = new JProperty("_" + property.Name, property.Value);
jToken.Replace(token);
}
}
if (jToken.HasValues)
{
//foreach won't work here as the call to jToken.Replace(token) above
//results in the collection modifed error.
for(int i = 0; i < jToken.Values().Count(); i++)
{
JToken obj = jToken.Values().ElementAt(i);
Process(obj);
}
}
}
This seemed to work well, prefixing numeric names with _. At this line:
XDocument document = JsonConvert.DeserializeXNode(jsonObject.ToString());
it crashed with an error saying that invalid/not well formed XML had been created. I don't have the actual error with me, but you can run the above code to replicate it.
I think from here you may need to revisit converting the JSON to XML in the first place. Is this a specific requirement?

c# dynamic json objects with dynamic names question

Before I get flagged for duplicate, I have the code from Dynamic json object with numerical keys working quite well now. The question with my numeric keys is that unfortunately, the JSON string I am getting is initially delimited by year, so would I use reflection to attempt to create a dynamic property on a dynamic object, and if so how? I know with a dynamic object I can't have obj["2010"] or obj[0]. In JavaScript this is no problem, just trying to get it working in C#. Ideas?
Example of JSON being returned:
{
"2010": [
{
"type": "vacation",
"alloc": "90.00"
},
Alternatively, sometimes the year is the second element as such:
I have no control over this json.
{
"year": [],
"2010": [
{
"type": "vacation",
"alloc": "0.00"
},

Maybe I'm misunderstanding your question, but here's how I'd do it:
static void Main(string[] args) {
var json = #"
{
'2010': [
{
'type': 'vacation',
'alloc': '90.00'
},
{
'type': 'something',
'alloc': '80.00'
}
]}";
var jss = new JavaScriptSerializer();
var obj = jss.Deserialize<dynamic>(json);
Console.WriteLine(obj["2010"][0]["type"]);
Console.Read();
}
Does this help?
I wrote a blog post on serializing/deserializing JSON with .NET: Quick JSON Serialization/Deserialization in C#

I have up-voted the question and JP's answer and am glad I dug around the internet to find this.
I have included a separate answer to simplify my use case for others to benefit from. The crux of it is:
dynamic myObj = JObject.Parse("<....json....>");
// The following sets give the same result
// Names (off the root)
string countryName = myObj.CountryName;
// Gives the same as
string countryName = myObj["CountryName"];
// Nested (Country capital cities off the root)
string capitalName = myObj.Capital.Name;
// Gives the same as
string capitalName = myObj["Capital"]["Name"];
// Gives the same as
string capitalName = myObj.Capital["Name"];
Now it all seems quite obvious but I just did not think of it.
Thanks again.