I have created a JSON file from JArray in MVC application from 2 coloumns of my Database. But there are 2 issues:
1. The format of the file. It has extra brackets for each object.
[
[
{
"Code": "1",
"Name": "ASSETS"
}
],
[
{
"Code": "2",
"Name": "LIABILITIES"
}
],
[
{
"Code": "3",
"Name": "CAPITAL"
}
],
[
{
"Code": "4",
"Name": "REVENUE"
}
]
]
I want it as:
[
{
"Code": "1",
"Name": "ASSETS"
},
{
"Code": "2",
"Name": "LIABILITIES"
},
{
"Code": "3",
"Name": "CAPITAL"
},
{
"Code": "4",
"Name": "REVENUE"
}
]
I have loaded the values in JArray from Database and then add it in another JArray. I am using 2 for loops. 1 for making JArray for 5000 entries. And the second for returning the next valid primary key from Database. But the problem is that, it takes more than 15 minutes to process the loops and return the JSON file. Why is there so much latency? And how can I make it fast. Here is the code.
int idd =0;
JArray Array = new JArray();
for (int b = 0; b<5000; b++)
{
idd = dbid(idd);
IEnumerable<MST> accList = new List<MST>
{
new MST
{
S1 = db.MSTs.Find(idd).S1,
S2 = db.MSTs.Find(idd).S2
}
};
JArray Arrayone = new JArray(
accList.Select(p => new JObject
{
{ "Code", p.S1 },
{ "Name", p.S2 },
})
);
Array.Add(Arrayone);
}
string jsonfile = JsonConvert.SerializeObject(Array,Formatting.Indented);
string path = #"C:\Users\Awais\Desktop\accounts.json";
System.IO.File.WriteAllText(path, jsonfile);
return View(v);
}
public int dbid(int id)
{
decimal i = db.MSTs.Max(a => a.N100);
MST m = new MST();
for (; id <= i; id++)
{
m = db.MSTs.Find(++id);
if (m == null)
continue;
else
{
break;
}
}
return id;
}
When I tried the first loop for 100 entries, it took about 60 seconds to return the file.
By modeling your JSON with anonymous type, you could've done it like this:
var array = (from coa in db.MSTs
select new { Code = coa.S2, Name = coa.S1 }).ToArray();
string jsonfile = JsonConvert.SerializeObject(array, Formatting.Indented);
string path = #"C:\Users\Awais\Desktop\accounts.json";
System.IO.File.WriteAllText(path, jsonfile);
It took me 319 ms for 100 000 objects.
Try:
public class RootObject
{
public string Code { get; set; }
public string Name { get; set; }
}
var o = new List<RootObject>();
for (var i = 0; i < 100; ++i)
{
o.Add(new RootObject
{
Code = "foo",
Name = "bar"
});
}
var v = JsonConvert.SerializeObject(o);
Took around 274 milliseconds for my list to Serialize.
I have removed all loops and instead try to solve it in query and making a single array.
var k = (from coa in db.MSTs
select new { S2 = coa.S2, S1 = coa.S1 }).ToList().
Select(x => new MST { S2 = x.S2, S1 = x.S1 }).ToList();
JArray Arrayone = new JArray(
k.Select(p => new JObject
{
{ "Code", p.S1 },
{ "Name", p.S2 },
})
);
string jsonfile = JsonConvert.SerializeObject(Arrayone,Formatting.Indented);
string path = #"C:\Users\Awais\Desktop\accounts.json";
System.IO.File.WriteAllText(path, jsonfile);
It solves both problems. i.e. Brackets formatting issue and latency. Now with this code it works in less than 5 seconds
Related
I have the below JSON where I'm trying to query the average confidence value and count of shape objects. I am using Newtonsoft to create the Json Object and to parse the Json Object. I'm gettiing error as "Cannot cast Newtonsoft.Json.Linq.JObject to Newtonsoft.Json.Linq.JToken". I understand I am treating object as array and hence the error, but I don't know how to treat the nested object. Please help.
{
"channel":{
"description": "James Newton-King\"s blog.",
"region":[
{
"title": "Json.NET 1.3 + New license + Now on CodePlex",
"description": "Announcing the release of Json.NET 1.3",
"link": "http://james.newtonking.com/projects/json-net.aspx",
"shape":{
"square":{
"type":"number",
"text":"$81.22",
"confidence":0.983
},
"circle":{
"type":"string",
"valueString":"50741890",
"text":"50741890",
"confidence":1.0
},
"rectangle":{
"type":"date",
"text":"01/01/2020",
"confidence":1.0
}
}
}
],
"errors":[
]
}
}
//My code
public void qryNode()
{
string json = File.ReadAllText(#"C:\Extract.json");
JObject rss = JObject.Parse(json);
var categories =
from c in rss["channel"]["region"].SelectMany(i => i["shape"]).Values<string>()
group c by c
into g
orderby g.Count() descending
select new { Category = g.Key, Count = g.Count() };
foreach (var c in categories)
{
Console.WriteLine(c.Category + " - Count: " + c.Count);
}
}
Once you have JObject parsed, you can get requested result like this:
var jObject = JObject.Parse(json);
var shapes = jObject["channel"]["region"]
.SelectMany(j => j["shape"]);
var confidences = shapes
.SelectMany(s => s.Select(i => i["confidence"]
.Value<float>()))
.ToList();
var result = new
{
ShapesCount = confidences.Count,
AverageConfidence = confidences.Average()
};
You can easily query with direct LINQ keywords like this
considering this JSON
{
"items": [
{
"id": "10",
"name": "one"
},
{
"id": "12",
"name": "two"
}
]
}
let's put it in a variable called json like this,
JObject json = JObject.Parse("{'items':[{'id':'10','name':'one'},{'id':'12','name':'two'}]}");
you can select all ids from the items where name is "one" using the following LINQ query
var Ids =
from item in json["items"]
where (string)item["name"] == "one"
select item["id"];
Then, you will have the result in an IEnumerable list
If dataGeneratorType is range then the value can be anything between dataGeneratorStart and dataGeneratorEnd. If dataGeneratorType is array with no length property then the value will be one value randomly selected from the array, otherwise it be two randomly selected values that equals to the length. If it is the object (which can be more nested) then it will again follow the above logic. But this is where it gets tricky for me. Is there any dynamic way to solve the problem in C#.
Input payload json
{
"temperature": {
"type": "int",
"dataGeneratorType": "range",
"dataGeneratorStart": -5,
"dataGeneratorEnd": 55
},
"salesAmount": {
"type": "float",
"dataGeneratorType": "array",
"dataGeneratorArray": [
0.51,
13.33,
20.01,
1.54
]
},
"city": {
"type": "string",
"dataGeneratorType": "array",
"dataGeneratorArray": [
"UK",
"Iceland",
"Portugal",
"Spain"
]
},
"relatedTags": {
"type": "array",
"dataGeneratorType": "array",
"dataGeneratorArray": [
"Sport",
"Hardware",
"Cycling",
"Magazines"
],
"length": 2
},
"salesDetail": {
"type": "object",
"dataGeneratorType": "object",
"dataGeneratorValue": {
"VAT": {
"type": "float",
"dataGeneratorType": "range",
"dataGeneratorStart": 0.0,
"dataGeneratorEnd": 20.0
},
"discountAmount": {
"type": "float",
"dataGeneratorType": "array",
"dataGeneratorArray": [
0.10,
0.15,
0.20
]
}
}
}
}
To output json:
{
"temperature": 20,
"salesAmount": 20.01,
"city": "Iceland",
"relatedTags": [
"Sport",
"Cycling"
],
"salesDetails": {
"VAT": 15.0,
"discount": 0.1
}
}
Please try this:
static string TransformJson(string inputPayload)
{
JObject obj = JObject.Parse(inputPayload);
var manipulatedObj = obj.DeepClone();
foreach (var child in obj)
{
var key = child.Key;
var t = child.Value["type"];
JToken genType;
if (!((JObject)child.Value).TryGetValue("dataGeneratorType", out genType))
{
continue; // genType is not found so, continue with next object.
}
var str = genType.Type == JTokenType.String ? genType.ToString().ToLower() : null;
switch (str)
{
case "range":
var r = new Random();
if (((string)t).ToLower() == "float")
{
var s = (float)child.Value["dataGeneratorStart"];
var e = (float)child.Value["dataGeneratorEnd"];
manipulatedObj[key] = r.NextDouble() * e;
}
else
{
var s = (int)child.Value["dataGeneratorStart"];
var e = (int)child.Value["dataGeneratorEnd"];
manipulatedObj[key] = r.Next(s, e);
}
break;
case "array":
var arr = child.Value["dataGeneratorArray"];
JToken lengthToken;
if (!((JObject)child.Value).TryGetValue("length", out lengthToken))
{
lengthToken = 2.ToString();
}
if ((string)t != "array")
{
manipulatedObj[key] = arr.OrderBy(a => a).LastOrDefault();
}
else
{
var count = arr.Count() >= (int)lengthToken ? (int)lengthToken : arr.Count();
var item = new JArray();
foreach (var m in Enumerable.Range(0, count).Select(i => arr[i]))
{
item.Add(m);
}
manipulatedObj[key] = item;
}
break;
case "object":
var transformJson = TransformJson(child.Value["dataGeneratorValue"].ToString());
manipulatedObj[key] = JObject.Parse(transformJson);
break;
default:
manipulatedObj[key] = child.Value["dataGeneratorValue"];
break;
}
}
return manipulatedObj.ToString(Formatting.Indented);
}
And use it as below:
var serializeObject = #"{""temperature"":{""type"":""int"",""dataGeneratorType"":""range"",""dataGeneratorStart"":-5,""dataGeneratorEnd"":55},""salesAmount"":{""type"":""float"",""dataGeneratorType"":""array"",""dataGeneratorArray"":[0.51,13.33,20.01,1.54]},""relatedTags"":{""type"":""array"",""dataGeneratorType"":""array"",""dataGeneratorArray"":[""Sport"",""Hardware"",""Cycling"",""Magazines""],""length"":2},""salesDetail"":{""type"":""object"",""dataGeneratorType"":""object"",""dataGeneratorValue"":{""VAT"":{""type"":""float"",""dataGeneratorType"":""range"",""dataGeneratorStart"":0.0,""dataGeneratorEnd"":20.0},""discountAmount"":{""type"":""float"",""dataGeneratorType"":""array"",""dataGeneratorArray"":[0.1,0.15,0.2]}}}}";
Console.WriteLine(serializeObject);
var outputJson = TransformJson(serializeObject);
Console.WriteLine(System.Environment.NewLine + "Modified Json = " + System.Environment.NewLine);
Console.WriteLine(outputJson);
I just added everything in one function but you can split into multiple functions and/or classes to make unit testing easier.
Here is the dotnet fiddle.
Here is the jist of the logic for your problem. To parse nested json you have to use a recursive statement and do the processing for each layer.
public Void Traverse(string myJsonString){
var jObj = JObject.Parse(jsonString);
foreach (var item in jObj)
{
if(item.Value["dataGeneratorType"].ToString().Equals("range")){
Console.WriteLine("Do some logic");
}
else if(item.Value["dataGeneratorType"].ToString().Equals("array")){
if(item.Value["length"] != null){
Console.WriteLine("Length is present do more logic");
}
else
{
Console.WriteLine("No length property present do more logic");
}
}
else if(item.Value["dataGeneratorType"].ToString().Equals("object")){
Console.WriteLine("It's an object");
Console.WriteLine(item.Value["dataGeneratorValue"]);
foreach (var nestedItem in item.Value["dataGeneratorValue"]){
Console.WriteLine("Nested Item");
Console.WriteLine(nestedItem);
//Recursive function call
Traverse(nestedItem.Value)//pass in as a json string
}
}
}
}
}
I have a Json as below
{
"name": "xxxx",
"type": "ccc",
"properties": {
"serialNumber": {
"value": "24-66292"
},
"documentLinks": {
"productManuals": {
"54868484": {
"url": {
"value": "xxxx"
},
"productName": {
"value": "R02400"
}
}
},
"keystringFiles": {
"60050588": {
"url": {
"value": "http://se-s-0010052.de.abb.com/stage/wc/!control.controller?action=view_document&doc_id=60050588"
},
"name": {
"value": "24-66292_160.kxt"
},
"fileSize": {
"value": 0.87
},
"addedDate": {
"value": "2012-01-19"
},
"addedBy": {
"value": "Loader"
}
}
}
}
},
"variables":{
"temperature" :{
"dataType": "number"
},
"modes" :{
"normal":{
"dataType": "string"
},
"fast":{
"dataType": "string"
}
}
}
}
I need to count the total number of elements in this excluding the root level elements. In this example it's like under "properties" below are the number of elements
serialNumber
documentLinks->productManuals->54868484->url
documentLinks->productManuals->54868484->productName
documentLinks->keystringFiles->60050588->url
documentLinks->keystringFiles->60050588->name
documentLinks->keystringFiles->60050588->fileSize
documentLinks->keystringFiles->60050588->addedDate
documentLinks->keystringFiles->60050588->addedBy
Under "variables"
temperature
modes->normal
modes->fast
Hence total number of elements is
8+3 = 11
I was trying multiple things as below, but I am not able to find the best logic which serves the purpose.
var ob = JObject.Parse(json);
var propCount = JObject.Parse(json).Root
.SelectTokens("properties")
.SelectMany(t => t.Children().OfType<JProperty>().Select(p => p.Name))
.ToArray();
var varCount = JObject.Parse(json).Root
.SelectTokens("variables")
.SelectMany(t => t.Children().OfType<JProperty>().Select(p => p.Name))
.ToArray();
int propCount = 0;
int variableCount = 0;
foreach (var prop in propCount)
{
propCount += ob["properties"][prop].Children().Count();
}
foreach (var variable in varCount)
{
variableCount += ob["variables"][variable].Children().Count();
}
var total = propCount + variableCount;
I think, based on what I understand, you have 11. You're missing properties->serialNumber in your count
private int GetLowestLevelCount(JToken obj)
{
int counter = 0;
if (obj.HasValues) // Checks if token has children.
{
foreach (var children in obj.Children())
{
counter += GetLowestLevelCount(children);
}
}
else // Lowest-level elem.
{
counter +=1;
}
return counter;
}
Use:
var ob = JObject.Parse(json);
var mainKeys = ob.Children().Select(x => ((JProperty)x).Name);
int lowestLevel = 0;
foreach (var mainKey in mainKeys)
{
var a = ob[mainKey];
if (a.HasValues) //Don't bother with top-level elements
{
foreach (var c in a.Children())
{
lowestLevel += GetLowestLevelCount(c);
}
}
}
Console.WriteLine(lowestLevel);
When I write this code:
string[] title= {"title1","title2","title3","title4"};
int[] iid= {124,122,333,234};
dynamic bucket = new JObject();
bucket.bucket = "Parking Lot";
bucket.roadmap = new JArray();
dynamic issue = new JObject();
dynamic json2 = new JArray(bucket);
for(int i=0; i<title.Length; i++) {
issue.title = title[i];
issue.iid = iid[i];
issue.order = i;
bucket.roadmap.Add(issue);
}
Console.WriteLine(json2.ToString());
My result is:
[
{
"bucket": "Parking Lot",
"roadmap": [
{
"title": "title4",
"iid": 234,
"order": 3
},
{
"title": "title2",
"iid": 122,
"order": 1
},
{
"title": "title3",
"iid": 333,
"order": 2
},
{
"title": "title4",
"iid": 234,
"order": 3
}
]
}
]
The first and last "roadmap" JObjects got repeated. There is a way if dynamically generate this JSON with their unique values I want to pass?
I also tried to do it non-dynamically.
JArray bucket = new JObject();
bucket.Add("bucket","Parking Lot");
bucket.Add("order", new JArray());
JObject issue = new JObject();
...
But I could not find a way to select "bucket.order.Add" to add issues to that JArray.
In the end, I am just trying to loop through a JArray with a bunch of data and I want to separate it from their bucket type, extract each issue value and store it in a new JSON, with the structure shown here.
Thanks
Create a new issue object before adding to the array.
for(int i=0; i<5; i++) {
issue = new JObject();
issue.title = "Elbow\"";
issue.iid = 123;
issue.order = i;
bucket.roadmap.Add(issue);
}
i need to fetch amount using following logic. if product.ID equals stores.ID then fetch product.amount
Json
{
"stores": [
{
"ID": 17736791,
"Name": "ABC"
},
{
"ID": 154423041,
"Name": "XYZ"
}
],
"product": [
{
"ID": 154423041,
"Amount": 19865337
}
]
}
i am using jtoken.selecttoken to fetch data as below. but it throws error as could not read query operator.
string path = ToJsonPath(product[ID=stores[*].ID].Amount);
var data= token.SelectTokens(path)
Updated, ToJsonPath
public string ToJsonPath(string query)
{
string normalizedQuery = query.Replace(DoubleQuotes, SingleQuotes);
StringBuilder jsonPath = new StringBuilder();
jsonPath.Append(string.Concat(RootElement, ChildOperator));
jsonPath.Append(normalizedQuery);
MatchCollection expressions = Regex.Matches(normalizedQuery, ExpressionRegexPattern);
StringBuilder expression = new StringBuilder();
for (int i = 0; i < expressions.Count; i++)
{
if (!Regex.IsMatch(expressions[i].Value, OperatorRegexPattern))
{
continue;
}
expression.Length = 0;
expression.Capacity = 0;
expression.Append(expressions[i].Value);
jsonPath.Replace(expression.ToString(), Placeholder);
string[] expressionTerms = expression.ToString()
.Split(new[] { AndOperator, OrOperator }, StringSplitOptions.RemoveEmptyEntries)
.Select(t => t.Trim())
.ToArray();
foreach (string expressionTerm in expressionTerms)
{
expression.Replace(expressionTerm, Placeholder);
expression.Replace(Placeholder, string.Concat(CurrentElement, ChildOperator, expressionTerm));
}
string expressionWithEscapedOperators = Regex.Replace(expression.ToString(), OperatorRegexPattern, " $& ");
string expressionWithDoubleEqualOperators = Regex.Replace(expressionWithEscapedOperators, EqualOperatorPattern, "$&$&");
string jsonExpression = string.Format(JsonExpressionTemplate, expressionWithDoubleEqualOperators);
jsonPath.Replace(Placeholder, jsonExpression);
}
return jsonPath.ToString();
}
Not sure about JSONPath but with LINQ to JSON this can be achieved as follows:
var obj = JObject.Parse(json);
var storeIds = obj["stores"]
.Select(s => (int)s["ID"])
.ToList();
var selectedAmount = obj["product"]
.Where(p => storeIds.Contains((int)p["ID"]))
.Select(p => (int)p["Amount"])
.FirstOrDefault();
Demo: https://dotnetfiddle.net/CRn5Az