What is the best way to convert json (or straightforward XML) to XML with namespaces (or with specific schema), without using strongly typed classes (C#)? (Using XSD, XSLT, template engine or other text based engine).
What is the most effective way (resources / performance)?
For example, to take the following object in json - string:
{
'item': {
'name': 'item #1'
'code': 'itm-123'
'image': {
'#url': 'http://www.foo.com/bar.jpg'
}
}
}
And convert it to:
<foo:item>
<foo:name>item #1</foo:name>
<foo:code>itm-123</foo:code>
<foo:image url="http://www.foo.bar"/>
</foo:item>
(The object can be more complex than the example above)
thanks
You could use json.net in order to do so.
Read this other post
It shows the other way round but should be pretty much the same.
With Cinchoo ETL, an open source library, this conversion can be done easily as below
string json = #"
{
'item': {
'name': 'item #1',
'code': 'itm-123',
'image': {
'#url': 'http://www.test.com/bar.jpg'
}
}
}";
StringBuilder xml = new StringBuilder();
using (var r = ChoJSONReader.LoadText(json))
{
using (var w = new ChoXmlWriter(xml)
.IgnoreRootName()
.IgnoreNodeName()
.WithDefaultXmlNamespace("foo", "http://temp.com")
)
{
w.Write(r);
}
}
Console.WriteLine(xml.ToString());
Output:
<foo:item xmlns:foo="http://temp">
<foo:name>item #1</foo:name>
<foo:code>itm-123</foo:code>
<foo:image url="http://www.test.com/bar.jpg" />
</foo:item>
Sample fiddle: https://dotnetfiddle.net/MITsuL
Disclaimer: I'm author of this library.
Related
Does anyone know how to convert the below nested JSON to CSV via CHOETL (An ETL framework for .NET)? Thank you!
I'm using this code but it will only return the first equipment record.
CODE:
{
using (var json = new ChoJSONReader("./test.json"))
{
csv.Write(json.Cast<dynamic>().Select(i => new
{
EquipmentId = i.GpsLocation.Equipment[0].EquipmentId,
InquiryValue = i.GpsLocation.Equipment[0].InquiryValue,
Timestamp = i.GpsLocation.Equipment[0].Timestamp
}));
}
}
JSON:
"GpsLocation": {
"Equipment": [
{
"EquipmentId": "EQ00001",
"InquiryValue": [
"IV00001"
],
"Timestamp": "2020-01-01 01:01:01.01",
},
{
"EquipmentId": "EQ00002",
"InquiryValue": [
"IV00002"
],
"Timestamp": "2020-01-01 01:01:01.01"
}
]
}
}````
As others suggest, the issue is you are only looking at the first element of the array.
It appears that the easiest way to control what you serialise into CSV is by correctly defining your source objects from JSON. JSON Path expressions come in pretty handy.
What I ended up doing here is query all JSON to return an array of Equipment objects regardless of where they are in the hierarchy (which means you may need to filter it a bit better depending on your full JSON).
Then it's pretty easy to define each field based on JSON path and just pass the result to CSVWriter.
Also check out some gotchas that I outlined in the respective comment lines.
void Main()
{
var jsonString = "{\"GpsLocation\":{\"Equipment\":[{\"EquipmentId\":\"EQ00001\",\"InquiryValue\":[\"IV00001\"],\"Timestamp\":\"2020-01-01 01:01:01.01\"},{\"EquipmentId\":\"EQ00002\",\"InquiryValue\":[\"IV00002\"],\"Timestamp\":\"2020-01-01 01:01:01.01\"}]}}";
var jsonReader = new StringReader(jsonString);
var csvWriter = new StringWriter(); // outputs to string, comment out if you want file output
//var csvWriter = new StreamWriter(".\\your_output.csv"); // writes to a file of your choice
using (var csv = new ChoCSVWriter(csvWriter))
using (var json = new ChoJSONReader(jsonReader)
.WithJSONPath("$..Equipment[*]", true) // firstly you scope the reader to all Equipment objects. take note of the second parameter. Apparently you need to pass true here as otherwise it just won't return anythig
.WithField("EquipmentId", jsonPath: "$.EquipmentId", isArray: false) // then you scope each field in the array to what you want it to be. Since you want scalar values, pass `isArray: false` for better predictability
.WithField("InquiryValue", jsonPath: "$.InquiryValue[0]", isArray: false) // since your InquiryValue is actually an array, you want to obtain first element here. if you don't do this, fields names and values would go askew
.WithField("Timestamp", jsonPath: "$.Timestamp", fieldType: typeof(DateTime), isArray: false)) // you can also supply field type, otherwise it seems to default to `string`
{
csv.WithFirstLineHeader().Write(json);
}
Console.WriteLine(csvWriter.GetStringBuilder().ToString()); // comment this out if writing to file - you won't need it
}
Update summary:
Pivoted to update the code to rely on JSON Path scoping - this seems to allow for field name manipulation with pretty low effort
Looking at your comment, you could probably simplify your file writer a little bit - use StreamWriter instead of StringWriter - see updated code for example
Here is the working sample of producing CSV from your JSON
string json = #"{
""GpsLocation"": {
""Equipment"": [
{
""EquipmentId"": ""EQ00001"",
""InquiryValue"": [
""IV00001""
],
""Timestamp"": ""2020-02-01 01:01:01.01"",
},
{
""EquipmentId"": ""EQ00002"",
""InquiryValue"": [
""IV00002""
],
""Timestamp"": ""2020-01-01 01:01:01.01""
}
]
}
}";
StringBuilder csv = new StringBuilder();
using (var r = ChoJSONReader.LoadText(json)
.WithJSONPath("$.GpsLocation.Equipment")
.WithField("EquipmentId")
.WithField("InquiryValue", jsonPath: "InquiryValue[0]", fieldType: typeof(string))
.WithField("Timestamp", fieldType: typeof(DateTime))
)
{
using (var w = new ChoCSVWriter(csv)
.WithFirstLineHeader())
w.Write(r);
}
Console.WriteLine(csv.ToString());
Output:
EquipmentId,InquiryValue,Timestamp
EQ00001,IV00001,2/1/2020 1:01:01 AM
EQ00002,IV00002,1/1/2020 1:01:01 AM
Sample fiddle: https://dotnetfiddle.net/hJWtqH
Your code is sound, but the issue is that you're only writing the first variable in the array by using i.GpsLocation.Equipment[0]. Instead, try looping over everything by putting it into a for loop, and changing the [0] to your iterating variable inside of said loop.
I have been searching and searching for a way to convert a json file to a csv and the vice versa using C#. I have searched google and have not come up with anything. Everything I've tried so far from the answers on stack overflow just do not work from me. Does anyone know of any tooling or tutorials I could have look at how to accomplish this with the .NET Framework? Usually I post what I've tried however I'm clearly far off here so it is pointless.
Compromises and Problems
You can accomplish this with the .NET Framework but there's not a clear and obvious way to just do this straight-up because of hierarchies and collections. What I mean by that is that CSV data is very flat and unstructured whereas JSON data is very organized and iterative. Let's take a simple chunk of JSON data that could look like this:
{
"Data": [
{
"Name":"Mickey Mouse",
"Friends":[ "Pluto", "Minnie", "Donald" ]
},
{
"Name":"Pluto",
"Friends":[ "Mickey" ]
}
]
}
The most obvious CSV file for that could be:
Name,Friend
Mickey Mouse,Pluto
Mickey Mouse,Minnie
Mickey Mouse,Donald
Pluto,Mickey
That's the easier conversion but let's say you just have that CSV file. It's not so obvious what the JSON should look like. One could argue that the JSON should look like this:
{
"Data": [
{ "Name":"Mickey Mouse", "Friend":"Pluto" },
{ "Name":"Mickey Mouse", "Friend":"Minnie" },
{ "Name":"Mickey Mouse", "Friend":"Donald" },
{ "Name":"Pluto", "Friend":"Mickey" },
]
}
That resulting JSON file is very different than the input JSON file. My point is that this isn't a simple/obvious conversion so any off-the-shelf or copy/paste solution will be imperfect. Whatever your solution is, you're going to have to make compromises or intelligent decisions.
.NET Framework Options
Now that we've gotten that out of the way, .NET gives you some capabilities to accomplish this out of the box and there are some good Nuget-supplied options as well. If you want to utilize pure .NET capabilities, you could use a combination of these two SO Answers:
Not perfect but this answer has some great code to get you started in the logic to generate a CSV file
This question and the resulting answers have some good info about generating JSON using just the .NET Framework and without any third-party utilities.
You should be able to apply the concepts in those two links PLUS the compromises and intelligent decisions you need to make from my first "Compromises and Problems" section of this post to accomplish what you need.
Something I've Done Before
I've done something similar where I actually used some functionality in the Microsoft.VisualBasic.FileIO namespace (works great in a C# app) in addition to Web API's serialization functionality to accomplish a CSV->JSON conversion using Dynamic objects (using the dynamic keyword) as an intermediary. The code is provided below. It's not terribly robust and makes some significant compromises but it has worked well for me. If you want to try this, you'll have to create your own version that goes in reverse, but as I mentioned in my first section, that's really the easy part.
using System.Collections.Generic;
using System.Dynamic;
using System.IO;
using System.Linq;
using System.Web.Http;
// NOTE: This is not purely my code. This was put together
// with the help of other SO questions that I wish I had the
// links to so I could credit them. You probably will find
// some chunk(s) of this code elsewhere on SO.
namespace Application1.Controllers
{
public class Foo
{
public string Csv { get; set; }
}
public class JsonController : ApiController
{
[HttpPost]
[Route("~/Csv/ToJson")]
public dynamic[] ConvertCsv([FromBody] Foo input)
{
var data = CsvToDynamicData(input.Csv);
return data.ToArray();
}
internal static List<dynamic> CsvToDynamicData(string csv)
{
var headers = new List<string>();
var dataRows = new List<dynamic>();
using (TextReader reader = new StringReader(csv))
{
using (var parser = new Microsoft.VisualBasic.FileIO.TextFieldParser(reader))
{
parser.Delimiters = new[] {","};
parser.HasFieldsEnclosedInQuotes = true;
parser.TrimWhiteSpace = true;
var rowIdx = 0;
while (!parser.EndOfData)
{
var colIdx = 0;
dynamic rowData = new ExpandoObject();
var rowDataAsDictionary = (IDictionary<string, object>) rowData;
foreach (var field in parser.ReadFields().AsEnumerable())
{
if (rowIdx == 0)
{
// header
headers.Add(field.Replace("\\", "_").Replace("/", "_").Replace(",", "_"));
}
else
{
if (field == "null" || field == "NULL")
{
rowDataAsDictionary.Add(headers[colIdx], null);
}
else
{
rowDataAsDictionary.Add(headers[colIdx], field);
}
}
colIdx++;
}
if (rowDataAsDictionary.Keys.Any())
{
dataRows.Add(rowData);
}
rowIdx++;
}
}
}
return dataRows;
}
}
}
If you want something more robust, then you can always leverage these great projects:
JSON.NET (This works VERY WELL with creating JSON from dynamic objects. Given that you're not using Web API, this would be the first place I would look to take the dynamic[] return value and convert it to JSON.)
CsvHelper
Besides using combination of multiple libraries to do the conversion of JSON to CSV and vice versa, Cinchoo ETL gives you unified interface to do the conversion between those 2 formats.
For a sample JSON file:
[
{
"Name" : "Xytrex Co.",
"Description" : "Industrial Cleaning Supply Company",
"AccountNumber" : "ABC15797531"
},
{
"Name" : "Watson and Powell, Inc.",
"Description" : "Law firm. New York Headquarters",
"AccountNumber" : "ABC24689753"
}
]
To produce CSV file:
Name,Description,AccountNumber
Xytrex Co.,Industrial Cleaning Supply Company,ABC15797531
Watson and Powell Inc.,Law firm. New York Headquarters,ABC24689753
JSON to CSV:
using (var p = ChoJSONReader.LoadText(json))
{
using (var w = new ChoCSVWriter(Console.Out)
.WithFirstLineHeader()
)
{
w.Write(p);
}
}
Sample fiddle: https://dotnetfiddle.net/T3u4W2
CSV to JSON:
using (var p = ChoCSVReader.LoadText(csv)
.WithFirstLineHeader()
)
{
using (var w = new ChoJSONWriter(Console.Out))
{
w.Write(p);
}
}
Sample fiddle: https://dotnetfiddle.net/gVlJVX
like Jaxidian mentioned, the problem is, that json can have a hierarchy, csv not.
So, there are two solutions I could suggest you:
create a hierarchical csv, shouldn't be much effort:
"Id";"Name";"Age";"Type"
"FriendId"
1;"Mickey Mouse";20;"mouse"
2
3
4
2;"Pluto";7;"dog"
1
3;"Minnie";20;"mouse"
4;"Donald";22;"duck"
create multiple files, could be more effort, but is more beautiful and more dynamic, when you eg. export from/import into database. Maybe this link could help you: http://www.snellman.net/blog/archive/2016-01-12-json-to-multicsv
all.csv (store all characters)
"Id";"Name";"Age";"Type"
1;"Mickey Mouse";20;"mouse"
2;"Pluto";7;"dog"
3;"Minnie";20;"mouse"
4;"Donald";22;"duck"
friends.csv (store all relations)
"FriendKey1";"FriendKey2"
1;2
2;1
1;3
1;4
I'm working with an API that is returning JSON.
I have a method that calls the api, and parses the JSON response for the desired nodes.
Up to this point everything has been working fine, except the latest JSON response appears to be malformed.
Other responses come back like:
{
"Keyword":"\"marhope\"",
"TermKey":null,
"Customers":[
{
"Memberships":[ ],
"CompanyId":0,
"ObjectId":112974,
"ObjectType":"Customer",
}
]
}
I use JObject.Parse to bring back the appropriate nodes by name.
The latest JSON response comes back as:
{
[
{
"AnimalId":9079117,
"SpeciesCode":"XX",
}
]
}
As you can see, there is no "name", and the JSON is slightly invalid.
How can I parse this. For the first example I was using the code below, but now that the JSON has no "name", I don't know how to approach this, thoughts?
JObject results = JObject.Parse(csr.SearchCustomer(1, 1, 870, term));
foreach (var resp in results["Customers"])
{
string obj = (string)resp["CompanyId"];
}
Jon Skeet is correct, the second JSON is invalid: you cannot have an array directly inside an object with no property name. The best course of action is to get the API developers to fix the JSON. However, if you're just looking for a quick and dirty workaround, you could strip off the the first and last brace from the invalid JSON and then parse it as an array using JArray.Parse.
string json = #"{
[
{
""AnimalId"":9079117,
""SpeciesCode"":""XX"",
}
]
}";
json = json.Substring(1, json.Length - 2);
JArray array = JArray.Parse(json);
foreach (JObject item in array.Children<JObject>())
{
Console.WriteLine("AnimalId: " + item["AnimalId"]);
Console.WriteLine("SpeciesCode: " + item["SpeciesCode"]);
}
Using the MongoDB C# driver how can I parse a JSON array (string) into BsonDocument[]?
We would like to store our mongo aggregation pipelines in separate JSON documents so need a way to parse them.
Not a bad idea if that suits your purposes. Yes the C# driver already supports BSON serialization from a JSON string source:
string json = '[
{ "$match: { "foo": "bar" } },
{ "$group": {
"_id": null,
"count": { "$sum": 1 }
}}
]';
BsonDocument pipeline =
MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonArray>(json);
So you can pull in your aggregation pipeline strings formatted as JSON and use them or manipulate them as BSON documents.
The accepted answer's result is a BsonArray, if you need a BsonDocument[] you'll do something like
BsonSerializer.Deserialize<BsonArray>(yourJsonString).Select(p => p.AsBsonDocument)
and if you need it as a List<BsonDocument>
BsonSerializer.Deserialize<BsonArray>(yourJsonString).Select(p => p.AsBsonDocument).ToList<BsonDocument>()
I'm having trouble converting a string of json facebook graph api, I used the facebook C# and json.Net.
But at conversion time it returns this error: Name can not begin with the '0 'character, hexadecimal value 0x30.
This is the code:
dynamic result = await _fb.GetTaskAsync ("me / feed");
FBxml JsonConvert.DeserializeXNode string = (result.ToString ()). ToString ();
It looks like there is a problem with portion of the json string as mentioned below (taken from your link http://jsfiddle.net/btripoloni/PaLC2/)
"story_tags": {
"0": [{
"id": "100000866891334",
"name": "Bruno Tripoloni",
"offset": 0,
"length": 15,
"type": "user"}]
},
Json cannot create class that begins with a numeric value such as '0'. Try creating the classes using the link http://json2csharp.com/ you will get an idea.
To solve this problem you can create a dynamic object and go through each properties OR create a JsonConverter and write your code in the ReadJson to convert the "0" to a meaningful name. May be this can help you http://blog.maskalik.com/asp-net/json-net-implement-custom-serialization
If this is not your problem then update the question with more information like class structure of FBxml, call stack of the exception (from which line of the json code is throwing the exception), Json version etc.
As keyr says, the problem is with those JSON properties that have numeric names. In XML names can contain numeric characters but cannot begin with one: XML (see the Well-formedness and error-handling section).
My idea was to recursively parse the JSON with JSON.Net, replacing properties that had numeric names:
var jsonObject = JObject.Parse(json);
foreach (var obj in jsonObject)
{
Process(obj.Value);
}
XDocument document = JsonConvert.DeserializeXNode(jsonObject.ToString());
....
private static void Process(JToken jToken)
{
if (jToken.Type == JTokenType.Property)
{
JProperty property = jToken as JProperty;
int value;
if (int.TryParse(property.Name, out value))
{
JToken token = new JProperty("_" + property.Name, property.Value);
jToken.Replace(token);
}
}
if (jToken.HasValues)
{
//foreach won't work here as the call to jToken.Replace(token) above
//results in the collection modifed error.
for(int i = 0; i < jToken.Values().Count(); i++)
{
JToken obj = jToken.Values().ElementAt(i);
Process(obj);
}
}
}
This seemed to work well, prefixing numeric names with _. At this line:
XDocument document = JsonConvert.DeserializeXNode(jsonObject.ToString());
it crashed with an error saying that invalid/not well formed XML had been created. I don't have the actual error with me, but you can run the above code to replicate it.
I think from here you may need to revisit converting the JSON to XML in the first place. Is this a specific requirement?