Create a tree node when serializing data - c#

I have some data queried from a SQL database and I use this code to serialize them:
List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
DataTable dt = new DataTable();
...
SqlDataAdapter adapt = new SqlDataAdapter();
adapt.Fill(dt);
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}
return JsonSerializer.Serialize(rows);
It gave me this result when I serialize them:
{
"operator": "Unknown",
"extrainfo": "potential client",
"Name": "John Doe",
"ID": 568910,
"LastUpdate": "2021-07-22T00:00:00",
"Interested?": "Yes",
"Does it have a valid contract?": "No",
"Contract type": "Prepaid",
"Client willing to pay more?": "Yes, up to 20%",
"Comments": {}
}
I want all data that comes after lastUpdate column to be serialized inside another node, which is simply called interview.
Here is how I want to serialize them:
{
"operator": "Unknown",
"extrainfo": "potential client",
"Name": "John Doe",
"ID": 568910,
"LastUpdate": "2021-07-22T00:00:00",
"interview": [
{
"question" : "Interested?",
"answer": "Yes"
},
{
"question" : "Does it have a valid contract?",
"answer": "No"
},
{
"question" : "Contract type",
"answer": "Prepaid"
},
{
"question" : "Client willing to pay more?",
"answer": "Yes, up to 20%"
},
{
"question" : "Comments",
"answer": ""
}
]
}
Here it's how a database row looks like:
I want some help on how to do this.

All data that comes after lastUpdate column to be serialized inside another node
After is relative:
Your DataTable might define the columns in a different order then they should present in the json
Serializer might use different ordering then your database schema
Filtering
I would suggest an approach where you list those fields that should be serialized as properties and treat the rest of them as interview question-answer pairs.
var propertyFields = new[] { "operator", "extrainfo", "Name", "ID", "LastUpdate" };
Capturing data
In order to create the required output (for interview) you might need to introduce a class or a struct. I've introduced a named ValueTuple to avoid creating such. But depending on your runtime environment it may or may not available. UPDATE: ValueTuples are not supported by System.Text.Json.JsonSerializer
struct Interview
{
[JsonPropertyName("question")]
public string Question { get; set; }
[JsonPropertyName("answer")]
public string Answer { get; set; }
}
Wire up
Let's put all this things together
static readonly string[] propertyFields = new[] { "operator", "extrainfo", "Name", "ID", "LastUpdate" };
...
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
var interview = new List<Interview>();
foreach (DataColumn col in dt.Columns)
{
string name = col.ColumnName;
object value = dr[col];
if (propertyFields.Contains(col.ColumnName))
row.Add(name, value);
else
interview.Add(new Interview { Question = name, Answer = value.ToString() });
}
row.Add("interview", interview);
rows.Add(row);
}

#admiri Please look serialisation example in this link
https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to?pivots=dotnet-5-0

I substituted a list of tuples for the sql data. For the purposes of the algorithm it's going to be the same.
Note that the simplest way to do this, is to create a POCO class to hold the actual values with the nested "inteview" POCO. If this is coming from SQL then you should know the column structure.
Giving your question, I'm going to make the assumption that for whatever reason that isn't possible and you don't know the column structure ahead of time and you're doing this on the fly.
In that case you're best bet is to not use any POCO classes - including the dictionary you're currently using - and simply write out the data as JSON. One way to do that is as follows:
static List<(string name, string[] values)> Data = new()
{
("operator", new[] { "Unknown" } ),
("extrainfo", new[] { "potential client" }),
("Name", new[] { "John Doe" }),
("ID", new[] { "568910" }),
("LastUpdate", new[] { "2021-07-22T00:00:00" }),
("Interested?", new[] { "Yes" } ),
("Does it have a valid contract?", new[] { "No" } ),
("Contract type", new[] { "Prepaid" } ),
("Client willing to pay more?", new[] { "Yes, up to 20%" } ),
("Comments", new string[] { }),
};
static string Serilize(List<(string name, string[] values)> data)
{
using var output = new MemoryStream();
using (var writer = new Utf8JsonWriter(output, new JsonWriterOptions() { Indented = true }))
{
bool foundQA = false;
writer.WriteStartObject();
foreach (var row in data)
{
if (!foundQA)
{
foreach (var value in row.values)
{
writer.WritePropertyName(row.name);
if (null != value)
writer.WriteStringValue(value);
else
writer.WriteStringValue("");
}
if (row.name == "LastUpdate")
{
writer.WritePropertyName("interview");
writer.WriteStartArray();
foundQA = true;
}
}
else
{
writer.WriteStartObject();
writer.WritePropertyName("question");
writer.WriteStringValue(row.name);
writer.WritePropertyName("answer");
writer.WriteStringValue(row.values.Length > 0 ? row.values[0] : "");
writer.WriteEndObject();
}
}
if (foundQA)
{
writer.WriteEndArray();
}
writer.WriteEndObject();
}
return Encoding.UTF8.GetString(output.ToArray());
}
static void Main(string[] args)
{
string formattedJson = Serilize(Data);
Console.WriteLine("Formatted output:");
Console.WriteLine(formattedJson);
}

Related

Inline string Json to formatted Json with C#

I have to send a variables Json to Mailgun but it only accepts the curly braces format when using multi level json files. So,
How can I pass from this:
{ "vehicle.type": "car"}
To this, with C#
{"vehicle": {"type": "car"}}
Having into consideration that sometimes it could be up to 3 nested elements. Like element1.element2.element3: value
Here is what I recommend.
note: I am using the Newtonsoft.Json library available via Nuget, if you are using .NET Core, you can use the built in System.Text.Json library.
Because we have multiple properties in the object with flattened property keys, qualified with .s and we need to convert these properties into a hierarchical, nested JSON structure, merging siblings appropriately at each level, a simple string replacement is neither safe nor effective.
Therefore, the approach here will be to parse the flattened property keys, such as "hospital.hospitalExtraData1.Street" recursively inferring and creating a hierarchy of nested objects.
Let's begin
var originalJson = #"{
""hospital.Name"": ""BestOneEver"",
""hospital.Estatus"": ""Active"",
""hospital.hospitalExtraData1.Street"": ""43"",
""hospital.hospitalExtraData1.Color"": ""Blue"",
""hospital.hospitalExtraData1.hospitalExtraData2.IsExpensive"": ""No"",
""hospital.hospitalExtraData1.hospitalExtraData2.Works24Hrs"": ""Yes"",
""patient.Name"": ""Leonel Messi"",
""patient.Age"": ""23""
}";
var original = JsonConvert.DeserializeObject<IDictionary<string, object>>(originalJson);
Now we have an object model we can work with and restructure.
We will do this using recursion
var original = JsonConvert.DeserializeObject<IDictionary<string, object>>(originalJson);
IDictionary<string, object> Expand(IDictionary<string, object> input)
{
var result = new Dictionary<string, object>();
foreach (var property in input)
{
var (key, remainder) = ParseKey(property.Key);
if (!result.ContainsKey(key))
{
result[key] = remainder != null
? Expand(new Dictionary<string, object>
{
[remainder] = property.Value
})
: property.Value;
}
else if (result[key] is IDictionary<string, object> inner)
{
inner[remainder] = property.Value;
result[key] = Expand(inner);
}
else
{
result[key] = property.Value;
}
}
return result;
}
(string key, string remainder) ParseKey(string key)
{
var dotIndex = key.IndexOf('.');
if (dotIndex != -1)
{
return (key.Substring(0, dotIndex), key.Substring(dotIndex + 1));
}
return (key, null);
}
var expanded = Expand(original);
var expandedJson = JsonConvert.SerializeObject(expanded, Newtonsoft.Json.Formatting.Indented);
Result:
{
"hospital": {
"Name": "BestOneEver",
"Estatus": "Active",
"hospitalExtraData1": {
"Street": "43",
"Color": "Blue",
"hospitalExtraData2": {
"IsExpensive": "No",
"Works24Hrs": "Yes"
}
}
},
"patient": {
"Name": "Leonel Messi",
"Age": "23"
}
}
Here is tricky way using string replacement.
replace for any dot(.) with this (": {") and add close tag (}) at the end for every dot(.) count. Good luck!
Try This:
IDictionary<string, object> Expand(IDictionary<string, object> d)
{
var result = new Dictionary<string, object>();
foreach (KeyValuePair<string, object> item in d)
{
var segments = item.Key.Split('.');
if (segments.Length > 1)
{
if (result.ContainsKey(segments[0]))
{
dynamic obj = new ExpandoObject();
obj = result[segments[0]];
IDictionary<string, object> myObj = obj;
myObj.Add(string.Join(".", segments.Skip(1)), item.Value);
result[segments[0]] = Expand(myObj);
}
else
{
result[segments[0]] = Expand(new Dictionary<string, object>
{
[string.Join(".", segments.Skip(1))] = item.Value
});
}
}
else
{
result[segments[0]] = item.Value;
}
}
return result;
}

JSON repeating inputs section from Excel

I have a specific JSON string that I need to match for a rest call. I'm pulling the data from an excel spreadsheet. One of the sections has repeating input like below. The data in my spreadsheet looks like this:
The JSON I need to generate looks like:
"detailInputs": [
{
"name": "SOGrid",
"repeatingInputs": [
{
"inputs": [
{
"name": "ItemNumber",
"value": "XYZ"
},
{
"name": "Quantity",
"value": "1"
}
]
},
{
"inputs": [
{
"name": "ItemNumber",
"value": "ABC"
},
{
"name": "Quantity",
"value": "3"
}
]
}
]
What I've tried so far is below (note jsonArraystring is the header information formatted in a previous section):
using (var conn = new OleDbConnection(connectionString))
{
sheetName = "Detail";
conn.Open();
var cmd = conn.CreateCommand();
cmd.CommandText = $"SELECT * FROM [{sheetName}$]";
using (var rdr = cmd.ExecuteReader())
{
var query = rdr.Cast<DbDataRecord>().Select(row => new {
name = row[0],
value = row[1],
//description = row[2]
});
var json = JsonConvert.SerializeObject(query);
jsonArrayString = jsonArrayString + ",\"detailInputs\":[{\"name\":\"SOGrid\",\"repeatingInputs\":[{\"inputs\": " + json + "}]}]}";
This is very close, but puts the "repeating Inputs" are all in one inputs section.
I also tried assigning the values to a dictionary and list in hopes of pulling the appropriate pairs and formatting the JSON from that, this is the beginning of that, but I'm not familiar enough with unraveling the key value pairs to get that formatted correctly.
using (var conn = new OleDbConnection(connectionString))
{
sheetName = "Detail";
conn.Open();
int counter = 0;
var cmd = conn.CreateCommand();
cmd.CommandText = $"SELECT * FROM [{sheetName}$]";
var values = new List<Dictionary<string, object>>();
var ListValues = new List<string>();
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
var fieldValues = new Dictionary<string, object>();
var fieldValuesList = new List<string>();
for (int i = 0; i < rdr.FieldCount; i++)
{
fieldValues.Add(rdr.GetName(i), rdr[i]);
fieldValuesList.Add(rdr.GetName(i));
}
// add the dictionary on the values list
values.Add(fieldValues);
}
The root question is how can I create a repeating inputs structure as shown in the JSON sample, by pulling from excel data.
What you want to do is to serialize the contents of the Excel worksheet as the array value of the "repeatingInputs" property, using a specific structure. I would suggest breaking this down into a series of LINQ transformations.
First, introduce a couple of extension methods:
public static class DataReaderExtensions
{
// Adapted from this answer https://stackoverflow.com/a/1202973
// To https://stackoverflow.com/questions/1202935/convert-rows-from-a-data-reader-into-typed-results
// By https://stackoverflow.com/users/3043/joel-coehoorn
public static IEnumerable<T> SelectRows<T>(this IDataReader reader, Func<IDataRecord, T> select)
{
while (reader.Read())
{
yield return select(reader);
}
}
}
public static class EnumerableExtensions
{
// Adapted from this answer https://stackoverflow.com/a/419058
// To https://stackoverflow.com/questions/419019/split-list-into-sublists-with-linq/
// By https://stackoverflow.com/users/50776/casperone
public static IEnumerable<List<T>> ChunkWhile<T>(this IEnumerable<T> enumerable, Func<List<T>, T, bool> shouldAdd)
{
if (enumerable == null || shouldAdd == null)
throw new ArgumentNullException();
return enumerable.ChunkWhileIterator(shouldAdd);
}
static IEnumerable<List<T>> ChunkWhileIterator<T>(this IEnumerable<T> enumerable, Func<List<T>, T, bool> shouldAdd)
{
List<T> list = new List<T>();
foreach (var item in enumerable)
{
if (list.Count > 0 && !shouldAdd(list, item))
{
yield return list;
list = new List<T>();
}
list.Add(item);
}
if (list.Count != 0)
{
yield return list;
}
}
}
The first method packages an IDataReader into an enumerable of typed objects, one for each row. Doing this makes it easier to feed the data reader's contents into subsequent LINQ transformations. The second method breaks a flat enumerable into an enumerable of "chunks" of lists, based on some predicate condition. This will be used to break the rows into chunks at each ItemNumber row.
Using these two extension methods we can generate the required JSON as follows:
public static string ExtractRows(string connectionString, string sheetName)
{
using (var conn = new OleDbConnection(connectionString))
{
conn.Open();
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = string.Format("SELECT * FROM [{0}]", sheetName);
using (var rdr = cmd.ExecuteReader())
{
var query = rdr
// Wrap the IDataReader in a LINQ enumerator returning an array of key/value pairs for each row.
// Project the first two columns into a single anonymous object.
.SelectRows(r =>
{
// Check we have two columns in the row, and the first (Name) column value is non-null.
// You might instead check that we have at least two columns.
if (r.FieldCount != 2 || r.IsDBNull(0))
throw new InvalidDataException();
return new { Name = r[0].ToString(), Value = r[1] };
})
// Break the columns into chunks when the first name repeats
.ChunkWhile((l, r) => l[0].Name != r.Name)
// Wrap in the container Inputs object
.Select(r => new { Inputs = r });
// Serialize in camel case
var settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
};
return JsonConvert.SerializeObject(query, Formatting.Indented, settings);
}
}
}
}
Which will generate the required value for "repeatingInputs":
[
{
"inputs": [
{
"name": "ItemNumber",
"value": "XYZ"
},
{
"name": "Quantity",
"value": "1"
}
]
},
{
"inputs": [
{
"name": "ItemNumber",
"value": "ABC"
},
{
"name": "Quantity",
"value": "3"
}
]
}
]

Convert json respone into datatable

I have a json response from API and I need to store that information in a data table. Response is something like :
{
"columns": [
"firstName",
"lastName",
"email",
"password",
],
"rows": [
[
"Alpha",
"Tango",
"AlphaTango#domain.com",
"123"
],
[
"Charle",
"Tango",
"CharlieTango#domain.com",
"456"
]
]
}
I need it to be converted into the format :
firstName lastName email password
Alpha Tango AlphaTango#domain.com 123
Charle Tango CharlieTango#domain.com 456
I did the reverse with the help of below snippet, but am not able to convert json back into data table acceptable format :
JsonConvert.SerializeObject(new {
columns = dt1.Columns.Cast<DataColumn>().Select(x => x.ColumnName),
rows = dt1.AsEnumerable().Select(r => r.ItemArray),
});
Any suggestion or pointers will be highly appreciated
For this case you have to do some manual iterations over your json data. Can you try the code below?
static void Main(string[] args)
{
string json = "{\"columns\":[\"firstName\",\"lastName\",\"email\",\"password\",],\"rows\":[[\"Alpha\",\"Tango\",\"AlphaTango#domain.com\",\"123\"],[\"Charle\",\"Tango\",\"CharlieTango#domain.com\",\"456\"]]}";
dynamic jObject = JObject.Parse(json);
//Get columns of your object
List<string> columns = JsonConvert.DeserializeObject<List<string>>(jObject.columns.ToString());
//Get rows of your object
List<string[]> rows = JsonConvert.DeserializeObject<List<string[]>>(jObject.rows.ToString());
using (DataTable dt = new DataTable())
{
//Create columns
foreach (string column in columns)
dt.Columns.Add(new DataColumn(column));
//Add rows
foreach (string[] row in rows)
{
int columnOrdinal = 0;
DataRow newRow = dt.NewRow();
foreach (string value in row)
{
newRow.SetField<string>(columnOrdinal, value);
columnOrdinal++;
}
dt.Rows.Add(newRow);
}
}
}
Result for the code above:
Hope this helps

How to serialize specific Json object from C#

I have a requirement where i need to serialize json object in below format
[{
"columns": [{
"title": "NAME"
}, {
"title": "COUNTY"
}],
"data": [
["John Doe", "Fresno"],
["Billy", "Fresno"],
["Tom", "Kern"],
["King Smith", "Kings"]
]
}]
Here i need to get this json object from two different source, one is Columns and other is data. Columns would come from a string which will be comma separated as
string columnNames = "Name, County";
and data would come from .net Datatable like
DataTable dt = new DataTable();
I tried with below code using JavaScriptSerializer but i am not able to format it in the required format. Actually, shared format is required to dynamically create jquery datatable. Here is my raw code in C#.
[WebMethod]
public static string ConvertDatadttoString(string appName)
{
DataTable dt = new DataTable();
dt.Columns.Add("Name", typeof(string));
dt.Columns.Add("County", typeof(string));
dt.Rows.Add("vo", "go.com");
dt.Rows.Add("pa", "pa.com");
System.Web.Script.Serialization.JavaScriptSerializer serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}
return serializer.Serialize(rows);
}
Above code is only serializing the DataTable and is not able to create in the required format. Thanks.
What I usually do, is I build a model based off of the data, and serialize that model.
This is how I'd imagine your model would look.
public class SampleClass
{
public IEnumerable<SampleItem> columns { get; set; }
public IEnumerable<IEnumerable<string>> data { get; set; }
}
public class SampleItem
{
public string title { get; set; }
}
And this is how I'd imagine you'd get the sample json
var sample = new List<SampleClass>
{
new SampleClass()
{
columns = new List<SampleItem>()
{
new SampleItem() {title = "NAME" },
new SampleItem() {title = "COUNTY" },
},
data = new List<List<string>>()
{
new List<string> { "John Doe", "Fresno" },
new List<string> { "Billy", "Fresno" },
new List<string> { "Tom", "Kern" },
new List<string> { "King Smith", "Kings" },
}
}
};
var serializer = new JavaScriptSerializer();
var json = serializer.Serialize(sample);
I'm sure you can figure out how to create that model based off of your real data. It's not that hard.
It's probably easier to create a class, but if you want to work with a Dictionary<string,object> then you need to first add an entry for your columns:
rows["columns"] = dt.Columns.Cast<DataTableColumn>()
.Select(c => new { title = c.ColumnName }).ToList();
And then you can add your data with something like:
rows["data"] = dt.Rows.Cast<DataRow>.Select(r => r.ItemArray).ToList();
Now you have a Dictionary<string,object> with two items columns and rows. columns contains a collection of objects with a property title and rows just contains an array of arrays for each row.
But this is a quick and dirty solution. I think creating a class as per #Sam I am's answer is cleaner and easier to maintain in the long run.
If you are starting with a comma separated list of column names, it really shouldn't be much harder to do. Something like:
var columns = columnNames.Split(","); // beware of column names that include commas!
row["columns"] = columns.Select(c => new { title = c });
row["data"] = dt.Rows.Cast<DataRow>.Select(r => columns.Select(c => r[c]).ToList());

SQL Data to JSON C#

LIST Time Status
A 22:05 0
B 22:10 1
C 22:30 1
A 22:40 0
C 22:50 1
B 22:60 1
The above table needs to be converted to below JSON format
[
{ "name": "A",
data: [ [22:05,0],
[22:40,0]
]
},
{ "name": "B",
data: [ [22:10,1],
[22:60,1]
]
}
{ "name": "C",
data: [ [22:30,1],
[22:50,1]
]
}
]
The above is in a DataTable , now need to format the JSON,the below code does not give me in the same format.
List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
Dictionary<string, object> row = null;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}
I would suggest reading the data from your table and creating a custom class to represent the data that needs to be serialized. You data would be represented in a class that looks like this:
public class MyType {
public string Name {get;set;}
public List<List<string>> Data {get;set;}
}
Then you would need a method to parse through your DataTable. Something like this might do it:
public List<MyType> ParseTable(DataTable dt) {
var myTypes = new List<MyType>();
var dictionary = new Dictionary<string, List<List<string>>>();
foreach(DataRow dr in dt.Rows) {
var name = dr[0];
var time = dr[1];
var status = dr[2];
if(!dictionary.ContainsKey(dr[0]) {
dictionary[name] = new List<List<string>>();
}
dictionary[name].Add(new List<string>{time, status});
}
foreach(var key = dictionary.Keys) {
myTypes.Add(new MyType {Name = key, Data = dictionary[key]});
}
return myTypes;
}
Then use a JSON serializer like http://james.newtonking.com/json to handle the actual serialization of your object into JSON.
That would look something like this:
public string Serialize(List<MyType> myTypes) {
return JsonConvert.SerializeObject(myTypes);
}
Please understand that this is freehand and off the cuff. So it may not be optimal and may need some tweaking. But this should get you where you are trying to go.

Categories