Deserialize Dynamic JSON C# - c#

I need to deserialize the following json:
{
"1": {
"oid": "46",
"order": "SD9999999999999",
"date": "2015-08-18 14:17:05",
"item": {
"0": {
"guid": "DEF456"
"price": "100.00"
},
"1": {
"guid": "ABC123",
"price": "99.99"
}
}
},
"2": {
"oid": "765",
"order": "SD0000000000000",
"date": "2015-08-18 14:17:05",
"item": {
"0": {
"guid": "GHI789"
"price": "10.00"
},
"1": {
"guid": "XYZ123",
"price": "9.99"
}
}
},
"store": 111,
"time": "2015-09-01 17:51:22"
}
The number of orders is unknown as well as the number of items in each order.
I have tried making a list of orders and dynamically looping through and JsonConvert.DeserializeObject() thhen adding each to the list, but this doesn't account for the items being dynamic and items end up equalling null. Any thoughts?


This works using JObject/JToken (after fixing the missing , in your json).
var json = System.IO.File.ReadAllText("test.json");
var jobj = Newtonsoft.Json.Linq.JObject.Parse(json);
//exclude "store" and "time"
var orders = jobj.Children().Where(x => x.Path != "store" && x.Path != "time").ToList();
//iterate over orders
foreach (var order in orders)
{
//"name" of order
var orderName = order.Path;
//and contents
var orderItems = order.First()["item"].ToList();
foreach (var item in orderItems)
{
var itemName = item.Path;
var itemContents = item.First();
var guid = (String)itemContents["guid"];
var price = Double.Parse((String)itemContents["price"]);
}
}
And for completeness, also code deserializing to a Dictionary<String, dynamic>
var dynObj = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<String, dynamic>>(json);
var dynOrders = dynObj.Where(x => x.Key != "store" && x.Key != "time").ToList();
foreach (dynamic order in dynOrders)
{
var orderName = order.Key;
var orderItems = order.Value.item;
foreach (var item in orderItems)
{
var itemName = item.Name;
var guid = (String)item.Value.guid;
var price = Double.Parse((String)item.Value.price);
}
}
I will say that for both of these, I had to set breakpoints and examine the objects and see what I had to work with and what would work to get that data. "Dictionary" style objects are more awkward in C# compared to arrays. I did find another way to do this via this answer. While the most verbose, also the most consistent across object levels.
//"force" to use IDictionary interface
IDictionary<string, JToken> dict = Newtonsoft.Json.Linq.JObject.Parse(json);
var dictOrders = dict.Where(x => x.Key != "store" && x.Key != "time").ToList();
foreach (var order in dictOrders)
{
var orderName = order.Key;
var orderProps = (IDictionary<string, JToken>)order.Value;
var oid = Int32.Parse((String)orderProps["oid"]);
var orderX = (String)orderProps["order"];
var date = DateTime.Parse((String)orderProps["date"]);
var orderItems = (IDictionary<string, JToken>)orderProps["item"];
foreach (var item in orderItems)
{
var itemName = item.Key;
var itemContents = (IDictionary<string, JToken>)item.Value;
var guid = (String)itemContents["guid"];
var price = Double.Parse((String)itemContents["price"]);
}
}
By using this pattern, you should be able to take any JToken, cast it to a IDictionary<string, JToken>, then deal with its contents in a consistent manner using Key for the property name and Value for the contents (which you would also cast to IDictionary so you can descent the object as far as you need).

Related

Sort multiple list inside List of objects with the same order of the first list

I Have this List of objects containing two lists of strings, that are linked I want to order the two lists valeurs and ProductUid
depending on the order of valeurs.
listProducts:
[
{
"groupBy": "coloris",
"valeurs": [
"Beige",
"Gris clair",
"Anthracite",
"Beige",
"Augusta",
"Venezia"
],
"ProductUid": [
"TEST1",
"TEST2",
"TEST3",
"TEST4",
"TEST5",
"TEST6"
]
},
{
"groupBy": "ref_commercial",
"valeurs": [
"29245",
"51625",
"25269",
"29245",
"72585",
"72584"
],
"ProductUid": [
"TEST1",
"TEST2",
"TEST3",
"TEST4",
"TEST5",
"TEST6"
]
}
]
So the final result will be like this.
SortedResult:
[
{
"groupBy": "coloris",
"valeurs": [
"Anthracite",
"Augusta",
"Beige",
"Beige",
"Gris clair",
"Venezia"
],
"ProductUid": [
"TEST3",
"TEST5",
"TEST1",
"TEST4",
"TEST2",
"TEST6"
]
},
{
"groupBy": "ref_commercial",
"valeurs": [
"25269",
"29245",
"29245",
"51625",
"72584",
"72585"
],
"ProductUid": [
"TEST3",
"TEST1",
"TEST4",
"TEST2",
"TEST6"
"TEST5",
]
}
]
What I already did is
While creating the listProduct
var result = lProd.SelectMany(x => x.Caracteristiques.Distinct(), (parentObj, childnum) =>
new
{
parentObj,
childnum
})
.GroupBy(x => x.childnum.nom)
.Select(x => new
{
groupBy = x.Key,
valeurs = x.Select(z => z.childnum.valeur).OrderBy(q => q), // Sort List Valeurs
ProductUid = x.Select(z => z.parentObj.ProductUid), // Want to do the same sort here
}).Where(sid => OrdredList.Any(si => si == sid.groupBy))
.OrderBy(x => OrdredList.IndexOf(x.groupBy));
I was able to sort the valeurs list but couldn't find a way to use the same sort on the productUID,
Is there a way to do it please ?

In order to make your task much easier I suggest you change the structure of the object to tuples of (property, uid). I would suggest doing this even if you didn't need to sort.
.Select(x => new
{
groupBy = x.Key,
valeursAndUids = x
.Select(z => (Valeur: z.childnum.valeur, Uid: z.parentObj.ProductUid))
.OrderBy(q => q.Valeur)
}

You can create a list of indices and sort that by looking up values in the first list, and then use the indices to select values from the second. Assuming the lists are of equal length:
var l1 = new[] {1, 0, 2, 3};
var l2 = new[] {"b", "a", "c", "d"};
var indices = new int[l1.Length];
for (int i = 0; i < indices.Length; i++)
{
indices[i] = i;
}
var sortedIndices = indices.OrderBy(i => l1[i]).ToList();
var l2Sorted = sortedIndices.Select(i => l2[i]).ToList();
var l1Sorted = sortedIndices.Select(i => l1[i]).ToList();
Console.WriteLine(string.Join(", ", l2Sorted));

This program produces the desired output:
var json = File.ReadAllText("data.json");
var list = JsonSerializer.Deserialize<List<X>>(json);
foreach(var x in list)
{
var combined = x.valeurs.Zip( x.ProductUid, (valuer, uid) => (valuer, uid) );
var sorted = combined.OrderBy(p => p.valuer);
x.valeurs = sorted.Select( p => p.valuer).ToList();
x.ProductUid = sorted.Select( p => p.uid).ToList();
// Or
// var combined = x.valeurs.Zip(x.ProductUid);
// var sorted = combined.OrderBy(p => p.First);
// x.valeurs = sorted.Select( p => p.First).ToList();
// x.ProductUid = sorted.Select( p => p.Second).ToList();
}
Console.WriteLine(JsonSerializer.Serialize(list, new JsonSerializerOptions()
{
WriteIndented = true
}));
class X {
public List<string> valeurs {get; set; }
public List<string> ProductUid {get; set; }
}

Here's a relatively simple way to achieve this:
var x = j.Select(e => {
var map = e.Valeurs.Select((v, i) => (v, i)).OrderBy(t => t.v).Select(t => t.i).ToArray();
return new SomeNamespace.SomeRoot
{
GroupBy = e.GroupBy,
Valeurs = map.Select(i => e.Valeurs[i]).ToArray(),
ProductUid = map.Select(i => e.ProductUid[i]).ToArray(),
};
}
);
j is an object representation of your original JSON. I'll give the classes that parse it at the end of the answer. In essence j is a 2-long array of SomeRoot, an object that has string GroupBy, string[] Valuers and string[] ProjectUids properties. It represents the objects inside your root json array
The first thing we do, on this line:
var map = e.Valeurs.Select((v, i) => (v, i)).OrderBy(t => t.v).Select(t => t.i).ToArray();
is, for each root object, project the e.Valuers to a tuple that includes the index at which the value is found at, i. Then we sort by the value v; this means we have a sorted array that remembers where the original element was found: Anthracite was at index 2 but after sorting the array has a tuple of (Anthracite, 2) as its first element. The whole array looks like:
[
(Anthracite, 2), <-- now in slot 0 but remembers it used to be in slot 2
(August, 4),
(Beige, 0),
(Beige, 3),
(Gris clair, 1)
(Venezia, 5)
]
This "memory of where it was" will be useful later. The next thing we do is throw the Anthracite etc away and just keep the 2 etc
This means that map is an array of ints that runs like [2,4,0,3,1,5]
If you access the original Valuers array in this order (2,4,0,3,1,5) you get the colors in order, Anthracite, August, Beige, Beige..
..and you can thus also use the same map array to access the ProjectUids to get those in that same order, 2,4,0,3,1,5
Hence your new arrays of Valuers and ProjectUids become:
Valeurs = map.Select(i => e.Valeurs[i]).ToArray(),
ProductUid = map.Select(i => e.ProductUid[i]).ToArray(),
You can think of it like *because map[0] == 2 then what was in index 2 in the original array is now in index 0 of the sorted array. The map array essentially maps "where it needs to be:where it actually is"
To change what you sort by, alter .OrderBy(t => t.v) - t is a tuple of v, the original Valuer value, and i, the index where it was found..
Here is the full code from your JSON to your requirement, including the classes for parsing your json:
//impl
var s =
#"[
{
""groupBy"": ""coloris"",
""valeurs"": [
""Beige"",
""Gris clair"",
""Anthracite"",
""Beige"",
""Augusta"",
""Venezia""
],
""ProductUid"": [
""TEST1"",
""TEST2"",
""TEST3"",
""TEST4"",
""TEST5"",
""TEST6""
]
},
{
""groupBy"": ""ref_commercial"",
""valeurs"": [
""29245"",
""51625"",
""25269"",
""29245"",
""72585"",
""72584""
],
""ProductUid"": [
""TEST1"",
""TEST2"",
""TEST3"",
""TEST4"",
""TEST5"",
""TEST6""
]
}
]";
var j = SomeNamespace.SomeRoot.FromJson(s);
var x = j.Select(e => {
var map = e.Valeurs.Select((v, i) => (v, i)).OrderBy(t => t.v).Select(t => t.i).ToArray();
return new SomeNamespace.SomeRoot
{
GroupBy = e.GroupBy,
Valeurs = map.Select(i => e.Valeurs[i]).ToArray(),
ProductUid = map.Select(i => e.ProductUid[i]).ToArray(),
};
}
);
//parsing classes
namespace SomeNamespace
{
using System;
using System.Collections.Generic;
using System.Globalization;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
public partial class SomeRoot
{
[JsonProperty("groupBy")]
public string GroupBy { get; set; }
[JsonProperty("valeurs")]
public string[] Valeurs { get; set; }
[JsonProperty("ProductUid")]
public string[] ProductUid { get; set; }
}
public partial class SomeRoot
{
public static SomeRoot[] FromJson(string json) => JsonConvert.DeserializeObject<SomeRoot[]>(json, SomeNamespace.Converter.Settings);
}
public static class Serialize
{
public static string ToJson(this SomeRoot[] self) => JsonConvert.SerializeObject(self, SomeNamespace.Converter.Settings);
}
internal static class Converter
{
public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings
{
MetadataPropertyHandling = MetadataPropertyHandling.Ignore,
DateParseHandling = DateParseHandling.None,
Converters =
{
new IsoDateTimeConverter { DateTimeStyles = DateTimeStyles.AssumeUniversal }
},
};
}
}

Combine list of json data based on ID

I have List of JArray like below trying to combine/concatenate the data based on ID
[{"ID":"A1","color1":"silver"}, {"ID": "A3", "color1": "Yellow"}]
[{"ID":"A1","color2":"blue","color3":"pink"}]
[{"ID":"A2","color4":"green"}]
[{"ID":"A1","color5":"red","color6":"aqua"}]
[{"ID":"A1","description":"looking for more colors"}]
Trying for an output like this
[{"ID":"A1",
"color1":"silver",
"color2":"blue",
"color3":"pink",
"color4": "",
"color5":"red",
"color6":"aqua",
"description":"looking for more colors"
},
{"ID":"A2",
"color1":"",
"color2":"",
"color3":"",
"color4": "green",
"color5":"",
"color6":"",
"description":""
},
{
"ID": "A3",
"color1": "yellow",
"color2":"",
"color3":"",
"color4": "",
"color5":"",
"color6":"",
"description":""
}
]
Any suggestion or help into the right direction please.
List<JArray> data = new List<JArray>();
JArray array = JArray.Parse(s); // s is [{"ID":"A1","color1":"silver"}]
data.Add(array);
for (int x = 0; x < data.Count() - 1; x++)
{
data[x].Merge(data[x + 1], new JsonMergeSettings
{
MergeArrayHandling = MergeArrayHandling.Union
});
var groupedByUserID = data[x].GroupBy(x => x["ID"]).ToList();
foreach (var item in groupedByUserID)
{
var firstToken = item.First();
var remainingToken = item.Values().Except(firstToken);
foreach (var i in remainingToken)
{
if (i.Type == JTokenType.Property)
firstToken[((JProperty)i).Name] = ((JProperty)i).Value;
}
jArray.Add(firstToken);
}
}
var result = jArray.ToString(Formatting.Indented);
Console.Write(result);

List<JArray> data = new List<JArray> {
JArray.Parse("[{ 'ID':'A1', 'color1':'silver' }, { 'ID':'A3', 'color1':'Yellow' }]"),
JArray.Parse("[{ 'ID':'A1', 'color2':'blue', 'color3':'pink' }]"),
JArray.Parse("[{ 'ID':'A2', 'color4':'green' }]"),
JArray.Parse("[{ 'ID':'A1', 'color5':'red', 'color6':'aqua' }]"),
JArray.Parse("[{ 'ID':'A1', 'description':'looking for more colors' }]")
};
var names = new HashSet<string>();
foreach (JObject x in data.SelectMany(arr => arr))
{
foreach (var kvp in x)
{
names.Add(kvp.Key);
}
}
var result = data
.SelectMany(arr => arr)
.GroupBy(token => token.Value<string>("ID"))
.Select(group =>
{
var obj = new JObject();
foreach (var name in names)
{
obj[name] = "";
}
foreach (JObject x in group)
{
foreach (var kvp in x)
{
obj[kvp.Key] = kvp.Value;
}
}
return obj;
});
var jArray = new JArray(result);
Console.WriteLine(jArray);

Query JSON Nested Object using LINQ

I have the below JSON where I'm trying to query the average confidence value and count of shape objects. I am using Newtonsoft to create the Json Object and to parse the Json Object. I'm gettiing error as "Cannot cast Newtonsoft.Json.Linq.JObject to Newtonsoft.Json.Linq.JToken". I understand I am treating object as array and hence the error, but I don't know how to treat the nested object. Please help.
{
"channel":{
"description": "James Newton-King\"s blog.",
"region":[
{
"title": "Json.NET 1.3 + New license + Now on CodePlex",
"description": "Announcing the release of Json.NET 1.3",
"link": "http://james.newtonking.com/projects/json-net.aspx",
"shape":{
"square":{
"type":"number",
"text":"$81.22",
"confidence":0.983
},
"circle":{
"type":"string",
"valueString":"50741890",
"text":"50741890",
"confidence":1.0
},
"rectangle":{
"type":"date",
"text":"01/01/2020",
"confidence":1.0
}
}
}
],
"errors":[
]
}
}
//My code
public void qryNode()
{
string json = File.ReadAllText(#"C:\Extract.json");
JObject rss = JObject.Parse(json);
var categories =
from c in rss["channel"]["region"].SelectMany(i => i["shape"]).Values<string>()
group c by c
into g
orderby g.Count() descending
select new { Category = g.Key, Count = g.Count() };
foreach (var c in categories)
{
Console.WriteLine(c.Category + " - Count: " + c.Count);
}
}

Once you have JObject parsed, you can get requested result like this:
var jObject = JObject.Parse(json);
var shapes = jObject["channel"]["region"]
.SelectMany(j => j["shape"]);
var confidences = shapes
.SelectMany(s => s.Select(i => i["confidence"]
.Value<float>()))
.ToList();
var result = new
{
ShapesCount = confidences.Count,
AverageConfidence = confidences.Average()
};

You can easily query with direct LINQ keywords like this
considering this JSON
{
"items": [
{
"id": "10",
"name": "one"
},
{
"id": "12",
"name": "two"
}
]
}
let's put it in a variable called json like this,
JObject json = JObject.Parse("{'items':[{'id':'10','name':'one'},{'id':'12','name':'two'}]}");
you can select all ids from the items where name is "one" using the following LINQ query
var Ids =
from item in json["items"]
where (string)item["name"] == "one"
select item["id"];
Then, you will have the result in an IEnumerable list

jtoken.selecttokens issue when comparing value of one array with another array

i need to fetch amount using following logic. if product.ID equals stores.ID then fetch product.amount
Json
{
"stores": [
{
"ID": 17736791,
"Name": "ABC"
},
{
"ID": 154423041,
"Name": "XYZ"
}
],
"product": [
{
"ID": 154423041,
"Amount": 19865337
}
]
}
i am using jtoken.selecttoken to fetch data as below. but it throws error as could not read query operator.
string path = ToJsonPath(product[ID=stores[*].ID].Amount);
var data= token.SelectTokens(path)
Updated, ToJsonPath
public string ToJsonPath(string query)
{
string normalizedQuery = query.Replace(DoubleQuotes, SingleQuotes);
StringBuilder jsonPath = new StringBuilder();
jsonPath.Append(string.Concat(RootElement, ChildOperator));
jsonPath.Append(normalizedQuery);
MatchCollection expressions = Regex.Matches(normalizedQuery, ExpressionRegexPattern);
StringBuilder expression = new StringBuilder();
for (int i = 0; i < expressions.Count; i++)
{
if (!Regex.IsMatch(expressions[i].Value, OperatorRegexPattern))
{
continue;
}
expression.Length = 0;
expression.Capacity = 0;
expression.Append(expressions[i].Value);
jsonPath.Replace(expression.ToString(), Placeholder);
string[] expressionTerms = expression.ToString()
.Split(new[] { AndOperator, OrOperator }, StringSplitOptions.RemoveEmptyEntries)
.Select(t => t.Trim())
.ToArray();
foreach (string expressionTerm in expressionTerms)
{
expression.Replace(expressionTerm, Placeholder);
expression.Replace(Placeholder, string.Concat(CurrentElement, ChildOperator, expressionTerm));
}
string expressionWithEscapedOperators = Regex.Replace(expression.ToString(), OperatorRegexPattern, " $& ");
string expressionWithDoubleEqualOperators = Regex.Replace(expressionWithEscapedOperators, EqualOperatorPattern, "$&$&");
string jsonExpression = string.Format(JsonExpressionTemplate, expressionWithDoubleEqualOperators);
jsonPath.Replace(Placeholder, jsonExpression);
}
return jsonPath.ToString();
}

Not sure about JSONPath but with LINQ to JSON this can be achieved as follows:
var obj = JObject.Parse(json);
var storeIds = obj["stores"]
.Select(s => (int)s["ID"])
.ToList();
var selectedAmount = obj["product"]
.Where(p => storeIds.Contains((int)p["ID"]))
.Select(p => (int)p["Amount"])
.FirstOrDefault();
Demo: https://dotnetfiddle.net/CRn5Az

Create JSON from 2 strings from Database (Latency and Formatting Issues)

I have created a JSON file from JArray in MVC application from 2 coloumns of my Database. But there are 2 issues:
1. The format of the file. It has extra brackets for each object.
[
[
{
"Code": "1",
"Name": "ASSETS"
}
],
[
{
"Code": "2",
"Name": "LIABILITIES"
}
],
[
{
"Code": "3",
"Name": "CAPITAL"
}
],
[
{
"Code": "4",
"Name": "REVENUE"
}
]
]
I want it as:
[
{
"Code": "1",
"Name": "ASSETS"
},
{
"Code": "2",
"Name": "LIABILITIES"
},
{
"Code": "3",
"Name": "CAPITAL"
},
{
"Code": "4",
"Name": "REVENUE"
}
]
I have loaded the values in JArray from Database and then add it in another JArray. I am using 2 for loops. 1 for making JArray for 5000 entries. And the second for returning the next valid primary key from Database. But the problem is that, it takes more than 15 minutes to process the loops and return the JSON file. Why is there so much latency? And how can I make it fast. Here is the code.
int idd =0;
JArray Array = new JArray();
for (int b = 0; b<5000; b++)
{
idd = dbid(idd);
IEnumerable<MST> accList = new List<MST>
{
new MST
{
S1 = db.MSTs.Find(idd).S1,
S2 = db.MSTs.Find(idd).S2
}
};
JArray Arrayone = new JArray(
accList.Select(p => new JObject
{
{ "Code", p.S1 },
{ "Name", p.S2 },
})
);
Array.Add(Arrayone);
}
string jsonfile = JsonConvert.SerializeObject(Array,Formatting.Indented);
string path = #"C:\Users\Awais\Desktop\accounts.json";
System.IO.File.WriteAllText(path, jsonfile);
return View(v);
}
public int dbid(int id)
{
decimal i = db.MSTs.Max(a => a.N100);
MST m = new MST();
for (; id <= i; id++)
{
m = db.MSTs.Find(++id);
if (m == null)
continue;
else
{
break;
}
}
return id;
}
When I tried the first loop for 100 entries, it took about 60 seconds to return the file.

By modeling your JSON with anonymous type, you could've done it like this:
var array = (from coa in db.MSTs
select new { Code = coa.S2, Name = coa.S1 }).ToArray();
string jsonfile = JsonConvert.SerializeObject(array, Formatting.Indented);
string path = #"C:\Users\Awais\Desktop\accounts.json";
System.IO.File.WriteAllText(path, jsonfile);
It took me 319 ms for 100 000 objects.

Try:
public class RootObject
{
public string Code { get; set; }
public string Name { get; set; }
}
var o = new List<RootObject>();
for (var i = 0; i < 100; ++i)
{
o.Add(new RootObject
{
Code = "foo",
Name = "bar"
});
}
var v = JsonConvert.SerializeObject(o);
Took around 274 milliseconds for my list to Serialize.

I have removed all loops and instead try to solve it in query and making a single array.
var k = (from coa in db.MSTs
select new { S2 = coa.S2, S1 = coa.S1 }).ToList().
Select(x => new MST { S2 = x.S2, S1 = x.S1 }).ToList();
JArray Arrayone = new JArray(
k.Select(p => new JObject
{
{ "Code", p.S1 },
{ "Name", p.S2 },
})
);
string jsonfile = JsonConvert.SerializeObject(Arrayone,Formatting.Indented);
string path = #"C:\Users\Awais\Desktop\accounts.json";
System.IO.File.WriteAllText(path, jsonfile);
It solves both problems. i.e. Brackets formatting issue and latency. Now with this code it works in less than 5 seconds

Categories