We Keep Coding

How to convert a json with nested properties to data table in c#

How can I convert a Json with nested properties into a DataTable for export as a csv file in a generic manner using c#. A sample json as follows:
{
"Length": {
"Value": 12.93,
"Unit": "m"
},
"Temperature": {
"Value": 28.32,
"Unit": "DegC"
},
"Color": "Blue"
}
which I would like to flatten out in a table as follows:
"Property" | "Value" | "Unit"
---------------------------------
"Length" | 12.93 | "m"
"Temperature" | 28.32 | "DegC"
"Color" | "Blue" |
Note that the Color property is not nested whereas the Length and Temperature properties are. In the case of a unit-less property such as Color nothing (or null) should be added to the "Unit" column in the data table.
Additionally the nested properties would mostly be of type double, but could contain any struct type.
Convert JSON to DataTable 's solution is the closest I have come, but of course does not work for the nested properties I have.

you can try this code
var jsonParsed = JObject.Parse(json);
var jsonArr = new JArray();
foreach (var prop in jsonParsed.Properties())
{
var jObj = new JObject { ["Property"] = prop.Name };
var valueType = prop.Value.GetType().Name;
if (valueType == "JValue")
{
jObj["Value"] = (string)prop.Value;
jObj["Unit"] = string.Empty;
}
else if (valueType == "JObject")
{
var nestedProp = (JObject)prop.Value;
jObj["Value"] = (string)nestedProp["Value"];
jObj["Unit"] = nestedProp["Unit"];
}
jsonArr.Add(jObj);
}
DataTable dataTable = jsonArr.ToObject<DataTable>();

I eventually solved it directly by creating a DataTable and populating it as follows:
DataTable dataTable = new();
DataColumn[] dataColumns = new DataColumn[3];
dataColumns[0] = new DataColumn("Property");
dataColumns[1] = new DataColumn("Value");
dataColumns[2] = new DataColumn("Unit");
dataTable.Columns.AddRange(dataColumns);
JToken json = JToken.Parse(jsonContent);
foreach (JProperty content in json)
{
DataRow row = dataTable.NewRow();
row["Property"] = content.Name;
if (content.Value.Type == JTokenType.Object)
{
row["Value"] = content.Value["Value"];
row["Unit"] = content.Value["Unit"];
}
else
{
row["Value"] = content.Value;
row["Unit"] = "-";
}
dataTable.Rows.Add(row);
}

Create a tree node when serializing data

I have some data queried from a SQL database and I use this code to serialize them:
List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
DataTable dt = new DataTable();
...
SqlDataAdapter adapt = new SqlDataAdapter();
adapt.Fill(dt);
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}
return JsonSerializer.Serialize(rows);
It gave me this result when I serialize them:
{
"operator": "Unknown",
"extrainfo": "potential client",
"Name": "John Doe",
"ID": 568910,
"LastUpdate": "2021-07-22T00:00:00",
"Interested?": "Yes",
"Does it have a valid contract?": "No",
"Contract type": "Prepaid",
"Client willing to pay more?": "Yes, up to 20%",
"Comments": {}
}
I want all data that comes after lastUpdate column to be serialized inside another node, which is simply called interview.
Here is how I want to serialize them:
{
"operator": "Unknown",
"extrainfo": "potential client",
"Name": "John Doe",
"ID": 568910,
"LastUpdate": "2021-07-22T00:00:00",
"interview": [
{
"question" : "Interested?",
"answer": "Yes"
},
{
"question" : "Does it have a valid contract?",
"answer": "No"
},
{
"question" : "Contract type",
"answer": "Prepaid"
},
{
"question" : "Client willing to pay more?",
"answer": "Yes, up to 20%"
},
{
"question" : "Comments",
"answer": ""
}
]
}
Here it's how a database row looks like:
I want some help on how to do this.

All data that comes after lastUpdate column to be serialized inside another node
After is relative:
Your DataTable might define the columns in a different order then they should present in the json
Serializer might use different ordering then your database schema
Filtering
I would suggest an approach where you list those fields that should be serialized as properties and treat the rest of them as interview question-answer pairs.
var propertyFields = new[] { "operator", "extrainfo", "Name", "ID", "LastUpdate" };
Capturing data
In order to create the required output (for interview) you might need to introduce a class or a struct. I've introduced a named ValueTuple to avoid creating such. But depending on your runtime environment it may or may not available. UPDATE: ValueTuples are not supported by System.Text.Json.JsonSerializer
struct Interview
{
[JsonPropertyName("question")]
public string Question { get; set; }
[JsonPropertyName("answer")]
public string Answer { get; set; }
}
Wire up
Let's put all this things together
static readonly string[] propertyFields = new[] { "operator", "extrainfo", "Name", "ID", "LastUpdate" };
...
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
var interview = new List<Interview>();
foreach (DataColumn col in dt.Columns)
{
string name = col.ColumnName;
object value = dr[col];
if (propertyFields.Contains(col.ColumnName))
row.Add(name, value);
else
interview.Add(new Interview { Question = name, Answer = value.ToString() });
}
row.Add("interview", interview);
rows.Add(row);
}

#admiri Please look serialisation example in this link
https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to?pivots=dotnet-5-0

I substituted a list of tuples for the sql data. For the purposes of the algorithm it's going to be the same.
Note that the simplest way to do this, is to create a POCO class to hold the actual values with the nested "inteview" POCO. If this is coming from SQL then you should know the column structure.
Giving your question, I'm going to make the assumption that for whatever reason that isn't possible and you don't know the column structure ahead of time and you're doing this on the fly.
In that case you're best bet is to not use any POCO classes - including the dictionary you're currently using - and simply write out the data as JSON. One way to do that is as follows:
static List<(string name, string[] values)> Data = new()
{
("operator", new[] { "Unknown" } ),
("extrainfo", new[] { "potential client" }),
("Name", new[] { "John Doe" }),
("ID", new[] { "568910" }),
("LastUpdate", new[] { "2021-07-22T00:00:00" }),
("Interested?", new[] { "Yes" } ),
("Does it have a valid contract?", new[] { "No" } ),
("Contract type", new[] { "Prepaid" } ),
("Client willing to pay more?", new[] { "Yes, up to 20%" } ),
("Comments", new string[] { }),
};
static string Serilize(List<(string name, string[] values)> data)
{
using var output = new MemoryStream();
using (var writer = new Utf8JsonWriter(output, new JsonWriterOptions() { Indented = true }))
{
bool foundQA = false;
writer.WriteStartObject();
foreach (var row in data)
{
if (!foundQA)
{
foreach (var value in row.values)
{
writer.WritePropertyName(row.name);
if (null != value)
writer.WriteStringValue(value);
else
writer.WriteStringValue("");
}
if (row.name == "LastUpdate")
{
writer.WritePropertyName("interview");
writer.WriteStartArray();
foundQA = true;
}
}
else
{
writer.WriteStartObject();
writer.WritePropertyName("question");
writer.WriteStringValue(row.name);
writer.WritePropertyName("answer");
writer.WriteStringValue(row.values.Length > 0 ? row.values[0] : "");
writer.WriteEndObject();
}
}
if (foundQA)
{
writer.WriteEndArray();
}
writer.WriteEndObject();
}
return Encoding.UTF8.GetString(output.ToArray());
}
static void Main(string[] args)
{
string formattedJson = Serilize(Data);
Console.WriteLine("Formatted output:");
Console.WriteLine(formattedJson);
}

Can I use a single json file to create a n number of instances of different classes?

Is possible to create (from the example below), lets say, 2 instances of Animal and 3 instances of Flower based on data from a single json file?
Example code:
class Nature
{
// do something with a specific json file
class Animal
{
string id;
bool isMammal;
}
class Flower
{
string id;
int numberOfPetals;
}
}
Expected result:
An x amount of instances of Animal
An y amount of instances of Flower
PS:
x and y depends on the data obtained from the json file.
The workaround I thought was to, instead of creating an json file, create a .txt file, which contains Json data fragments. Then, save the contents of the .txt file into a variable. And finally, from that variable select each json data fragments to work with them individually, as if each one were an individual json file.
But, would it have a simpler way of doing this?

You can use JSON structure instead of maintining the count.
{
"AnimalCollection" :
[
{ "id":"Animal-1", "IsMammal":true },
{ "id":"Animal-2", "IsMammal":true },
{ "id":"Animal-3", "IsMammal":true }
],
"FlowerCollection":
[
{ "id":"Flower-1", "numberOfPetals":30 },
{ "id":"Flower-2", "numberOfPetals":20 },
{ "id":"Flower-3", "numberOfPetals":10 },
{ "id":"Flower-4", "numberOfPetals":3 }
]
}
Then you can deserialize this using newtonsoft.json into below type
public class Data
{
public Animal[] AnimalCollection {get;set;}
public Flower[] FlowerCollection{get;set;
}
It will contain 3 animal instances and 4 flower instances from JSON.
Hope this helps.

You can load the JSON file and convert the data into datatable. Create new list of Animal and Flower.
c# code as follows:
//Declare typed list of Animals
List<Animal> AllAnimals = new List<Animal>();
//Declare typed list of Flowers
List<Flower> AllFlowers = new List<Flower>();
DataTable dt;
//Load JSON file data
using (StreamReader r = new StreamReader(JSONFilePath))
{
string json = r.ReadToEnd();
//convert JSON data to datatable
dt = (DataTable)JsonConvert.DeserializeObject(json, (typeof(DataTable)));
}
foreach(DataRow row in dt.Rows)
{
if (row[1].ToString() == "Animal")
{
Animal NewAnimal = new Animal();
NewAnimal.id = row[0].ToString();
NewAnimal.isMammal = row[2].ToString();
AllAnimals.Add(NewAnimal);
}
else
{
Flower NewFlower = new Flower();
NewFlower.id = row[0].ToString();
NewFlower.numberOfPetals = row[3].ToString();
AllFlowers.Add(NewFlower);
}
}
Below is a sample of json data that can be loaded from a file:
{
{
"id": "0",
"EntryType": "Animal",
"IsMamal": "True",
"numberOfPetals": ""
},
{
"id": "1",
"EntryType": "Animal",
"IsMamal": "True",
"numberOfPetals": ""
},
{
"id": "2",
"EntryType": "Flower",
"IsMamal": "",
"numberOfPetals": "8"
},
{
"id": "1",
"EntryType": "Flower",
"IsMamal": "",
"numberOfPetals": "6"
},
{
"id": "2",
"EntryType": "Flower",
"IsMamal": "",
"numberOfPetals": "10"
}
}

How to serialize specific Json object from C#

I have a requirement where i need to serialize json object in below format
[{
"columns": [{
"title": "NAME"
}, {
"title": "COUNTY"
}],
"data": [
["John Doe", "Fresno"],
["Billy", "Fresno"],
["Tom", "Kern"],
["King Smith", "Kings"]
]
}]
Here i need to get this json object from two different source, one is Columns and other is data. Columns would come from a string which will be comma separated as
string columnNames = "Name, County";
and data would come from .net Datatable like
DataTable dt = new DataTable();
I tried with below code using JavaScriptSerializer but i am not able to format it in the required format. Actually, shared format is required to dynamically create jquery datatable. Here is my raw code in C#.
[WebMethod]
public static string ConvertDatadttoString(string appName)
{
DataTable dt = new DataTable();
dt.Columns.Add("Name", typeof(string));
dt.Columns.Add("County", typeof(string));
dt.Rows.Add("vo", "go.com");
dt.Rows.Add("pa", "pa.com");
System.Web.Script.Serialization.JavaScriptSerializer serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}
return serializer.Serialize(rows);
}
Above code is only serializing the DataTable and is not able to create in the required format. Thanks.

What I usually do, is I build a model based off of the data, and serialize that model.
This is how I'd imagine your model would look.
public class SampleClass
{
public IEnumerable<SampleItem> columns { get; set; }
public IEnumerable<IEnumerable<string>> data { get; set; }
}
public class SampleItem
{
public string title { get; set; }
}
And this is how I'd imagine you'd get the sample json
var sample = new List<SampleClass>
{
new SampleClass()
{
columns = new List<SampleItem>()
{
new SampleItem() {title = "NAME" },
new SampleItem() {title = "COUNTY" },
},
data = new List<List<string>>()
{
new List<string> { "John Doe", "Fresno" },
new List<string> { "Billy", "Fresno" },
new List<string> { "Tom", "Kern" },
new List<string> { "King Smith", "Kings" },
}
}
};
var serializer = new JavaScriptSerializer();
var json = serializer.Serialize(sample);
I'm sure you can figure out how to create that model based off of your real data. It's not that hard.

It's probably easier to create a class, but if you want to work with a Dictionary<string,object> then you need to first add an entry for your columns:
rows["columns"] = dt.Columns.Cast<DataTableColumn>()
.Select(c => new { title = c.ColumnName }).ToList();
And then you can add your data with something like:
rows["data"] = dt.Rows.Cast<DataRow>.Select(r => r.ItemArray).ToList();
Now you have a Dictionary<string,object> with two items columns and rows. columns contains a collection of objects with a property title and rows just contains an array of arrays for each row.
But this is a quick and dirty solution. I think creating a class as per #Sam I am's answer is cleaner and easier to maintain in the long run.
If you are starting with a comma separated list of column names, it really shouldn't be much harder to do. Something like:
var columns = columnNames.Split(","); // beware of column names that include commas!
row["columns"] = columns.Select(c => new { title = c });
row["data"] = dt.Rows.Cast<DataRow>.Select(r => columns.Select(c => r[c]).ToList());

SQL Data to JSON C#

LIST Time Status
A 22:05 0
B 22:10 1
C 22:30 1
A 22:40 0
C 22:50 1
B 22:60 1
The above table needs to be converted to below JSON format
[
{ "name": "A",
data: [ [22:05,0],
[22:40,0]
]
},
{ "name": "B",
data: [ [22:10,1],
[22:60,1]
]
}
{ "name": "C",
data: [ [22:30,1],
[22:50,1]
]
}
]
The above is in a DataTable , now need to format the JSON,the below code does not give me in the same format.
List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
Dictionary<string, object> row = null;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}

I would suggest reading the data from your table and creating a custom class to represent the data that needs to be serialized. You data would be represented in a class that looks like this:
public class MyType {
public string Name {get;set;}
public List<List<string>> Data {get;set;}
}
Then you would need a method to parse through your DataTable. Something like this might do it:
public List<MyType> ParseTable(DataTable dt) {
var myTypes = new List<MyType>();
var dictionary = new Dictionary<string, List<List<string>>>();
foreach(DataRow dr in dt.Rows) {
var name = dr[0];
var time = dr[1];
var status = dr[2];
if(!dictionary.ContainsKey(dr[0]) {
dictionary[name] = new List<List<string>>();
}
dictionary[name].Add(new List<string>{time, status});
}
foreach(var key = dictionary.Keys) {
myTypes.Add(new MyType {Name = key, Data = dictionary[key]});
}
return myTypes;
}
Then use a JSON serializer like http://james.newtonking.com/json to handle the actual serialization of your object into JSON.
That would look something like this:
public string Serialize(List<MyType> myTypes) {
return JsonConvert.SerializeObject(myTypes);
}
Please understand that this is freehand and off the cuff. So it may not be optimal and may need some tweaking. But this should get you where you are trying to go.