We Keep Coding

C# Create Deedle DataFrame from JSON response

I'm having a bit of trouble loading a JSON response from this request into a Deedle DataFrame:https://sampleserver6.arcgisonline.com/arcgis/rest/services/Earthquakes_Since1970/FeatureServer/0/query?where=OBJECTID%3C10&returnGeometry=false&f=json
In the JSON, what I'm interested are the features. More specifically, for each feature there are attributes - I want essentially just a collection of those attributes to load into a DataFrame. In this particular case, there is only one attribute "name" so my expectation is that the resulting DataFrame would have a column "name" with the values shown.
I've tried using json2csharp and creating my own class, but the result either doesn't have the column header/values or the values are missing. I'm not really sure what I'm doing wrong or if I'm even approaching this the right way. My understanding from the Deedle documentation is that it should be possible to create a DataFrame from a collection of objects: https://bluemountaincapital.github.io/Deedle/csharpframe.html#Creating-and-loading-data-frames. Certainly, using the Enumerable example listed on the page works as expected.
Here is the pertinent section of my code:
string url = "https://sampleserver6.arcgisonline.com/arcgis/rest/services/Earthquakes_Since1970/FeatureServer/0/query?";
WebClient wc = new WebClient();
wc.QueryString.Add("where", "OBJECTID<10");
wc.QueryString.Add("returnGeometry", "false");
wc.QueryString.Add("f", "json");
var data = wc.UploadValues(url, "POST", wc.QueryString);
var responseString = UnicodeEncoding.UTF8.GetString(data);
JObject o = JObject.Parse(responseString);
dynamic x = JsonConvert.DeserializeObject(responseString);
var testObjList = new List<dynamic>();
foreach (dynamic element in x.features)
{
testObjList.Add(new myClass { name = element.attributes.name});
Console.WriteLine($"{element.attributes.name}");
}
var dfObjects = Frame.FromRecords(testObjList);
dfObjects.Print();
var df = Frame.FromRecords(test);
df.Print(); // No headers or values shown
where myClass is just this:
public class myClass{
public string name { get; set; }
}
Any help/pointers would be much appreciated!

The Frame.FromRecords operation relies on static type information to figure out what properties the class has. In your case, you define the list of objects as List<dynamic> - this is compiled as Object and so Deedle does not see any members.
To fix this, all you need to do is to define the type as a list of myClass objects:
var testObjList = new List<myClass>();
A more compact approach using an anonymous type would work too:
var testObjList =
((IEnumerable<dynamic>)x.features).Select(element =>
new { name = element.attributes.name });
var dfObjects = Frame.FromRecords(testObjList);

JSON Data to dynamic (Anonymous type) conversion

I'm having trouble converting JSON data to a dynamic type definition. I've looked at JObject, JsonConvert serialize/deserialize and nothing works. The closest thing to making this work is JsonConvert.DeserializeAnonymousType but this requires a definition that matches the JSON. In my case, the JSON is quite complicated so I really need to convert an instance of the JSON to a fully anonymous, dynamic type.
dynamic rawjson = #"{ 'tags': { 'abcd' : '12345' },'properties': { 'desired': { 'PropOne' : '2345', 'PropTwo' : '6789' } } }";
#region reference object
dynamic reference = new
{
SomeName = $"xxxx",
initialTwin = new
{
tags = new { abcd = 12345 },
properties = new
{
desired = new
{
PropOne = "2345",
PropTwo = "6789"
}
}
}
};
#endregion
dynamic anonobject = JsonConvert.DeserializeAnonymousType(rawjson, reference.initialTwin);
dynamic testobject = new
{
SomeName = $"xxxx",
initialTwin = $"{anonobject}"
};
I need the "testobject" in the code above to look exactly like the "reference" object. Using the DesializeAnonymousType gets me very close but the definition would be very hard to create and maintain.
How can I get the same results working from an instance of JSON data without typing the definition for DeserializeAnonyousType? Is that possible?

I have tested the way below and it worked for me:
var anonobject = JsonConvert.DeserializeObject<ExpandoObject>(rawjson);
dynamic testobject = new
{
SomeName = $"xxxx",
initialTwin = anonobject
};
But since this is a dynamic object, you need to know the properties you may want to use.
Example:
Console.WriteLine(testobject.initialTwin.tags.abcd);

Parsing json files of different schemas to different tables json.net

I'm receiving json files of different schema and have to dump them in sql data base.
The json files have the schema
{'type':'abc','data':{'column1':'x','column2':'y',.........}}
Corresponding to each type of schema I have a strongly typed class named similar to the type but with word 'Table' attached..
eg. 'abcTable' which has only the schema of json.data (column1, column2, ...)
So, what I can do is do a dynamic deserializing of the main json and then based on the type value do a strongly typed json parsing of the corresponding data
dynamic jsondata = JsonConvert.DeserializeObject<dynamic>(json);
if (jsonata.type=='abc')
{
var abcobj = JsonConvert.DeserializeObject<abcTable>(jsondata.data);
}
Here I'm deserializing the object twice, so don't look like the right way of doing..
Also I have 25+ such schemas and a similar number of classes/tables
So, I will be using a lot of if / else if /else statements...
I would like to understand if there are other better ways of solving what I'm trying to do..
Any help is sincerely appreciated..
Thanks

As usually, JObject is your friend:
var parsed = JObject.Parse(json);
var type = parsed.Value<string>("type");
if (type == "abc")
{
var abcObject = parsed["data"].ToObject<abcTable>();
}
I order to avoid many ifs, you can use the follwing pattern:
public interface ITableType
{
bool Match(string type);
void Handle(JToken jsonTable);
}
public AbcTableHandler: ITableType
{
public bool Match(string type)
{
return type == "abc";
}
public void Handle(JToken jsonTable)
{
var abcTable = jsonTable.ToObject<abcTable>();
// other code
}
}
usage:
var handlers = new[] { new AbcTableHandler() };
// ...
var parsed = JObject.Parse(json);
var type = parsed.Value<string>("type");
var handler = handlers.SingleOfDefault(h => h.Match(type));
if (handler == null)
throw new InvalidOperationException("Cannot find handler for " + type);
handler.Handle(parsed["data"]);
EDIT:
Adding multiple handlers:
var handlers = new ITableType[] { new AbcTableHandler(), new OtherHandler, etc.. };
or
var handlers = new List<ITableType>();
handlers.Add(new AbcTableHandler());
handlers.Add(new OtherHandler());

C# deserialization

How to deserialize with keyvaluepair the above json
string stations = [{"2":false,"1":"Aforre","0":"WS6"},{"2":false,"1":"Alodtau","0":"WS3"},{"2":false,"1":"Balimo Station","0":"WS36"}]
I what like this
var get = js.Deserialize<Dictionary<string,dynamic>>(stations);

try this:
string stations = "[{'2':false,'1':'Aforre','0':'WS6'},{'2':false,'1':'Alodtau','0':'WS3'},{'2':false,'1':'Balimo Station','0':'WS36'}]";
var serializer = new JavaScriptSerializer();
dynamic value = serializer.DeserializeObject(stations);
and you can access objects like:
var a = value[0]["0"];
and a will have "WS6" (according to your JSON)

The JSON shown is an array. You might try desetializing to:
Dictionary<string, object>[]
i.e.
var get = js.Deserialize<Dictionary<string,object>[]>(stations);

One of the best option is create a class that consist of all keyvalue pair and then deserialize json string to the object of created class

You can try using dynamic variable as following:
string stations = "[{'2':false,'1':'Aforre','0':'WS6'},{'2':false,'1':'Alodtau','0':'WS3'},{'2':false,'1':'Balimo Station','0':'WS36'}]";
var serializer = new JavaScriptSerializer();
dynamic serializevalues = serializer.DeserializeObject(stations);
var valueof1 = serializevalues[0]["1"];
Response.Write(valueof1);
The above will print output "Aforre'.

Find differences between two json objects

Are there any libraries in .Net to help compare and find differences between two json objects? I've found some solutions available for JavaScript, but nothing interesting for C#. The point of my question is to create json with changes marked in some way, based on the comparison. So that the user could see where the changes are.

using Microsoft.XmlDiffPatch;
using Newtonsoft.Json;
Convert each json to xml and use MS XmlDiff libary. Available on nuget. Differences are given in another xml doc which here I write to the console. This is suitable for unit testing for example.
public bool CompareJson(string expected, string actual)
{
var expectedDoc = JsonConvert.DeserializeXmlNode(expected, "root");
var actualDoc = JsonConvert.DeserializeXmlNode(actual, "root");
var diff = new XmlDiff(XmlDiffOptions.IgnoreWhitespace |
XmlDiffOptions.IgnoreChildOrder);
using (var ms = new MemoryStream())
using (var writer = new XmlTextWriter(ms, Encoding.UTF8))
{
var result = diff.Compare(expectedDoc, actualDoc, writer);
if (!result)
{
ms.Seek(0, SeekOrigin.Begin);
Console.WriteLine(new StreamReader(ms).ReadToEnd());
}
return result;
}
}

I have used different JSON objects than those in your example but it will apply to your case correctly.
private static string GetJsonDiff(string action, string existing, string modified, string objectType)
{
// convert JSON to object
JObject xptJson = JObject.Parse(modified);
JObject actualJson = JObject.Parse(existing);
// read properties
var xptProps = xptJson.Properties().ToList();
var actProps = actualJson.Properties().ToList();
// find differing properties
var auditLog = (from existingProp in actProps
from modifiedProp in xptProps
where modifiedProp.Path.Equals(existingProp.Path)
where !modifiedProp.Value.ToString().Equals(existingProp.Value.ToString())
select new AuditLog
{
Field = existingProp.Path,
OldValue = existingProp.Value.ToString(),
NewValue = modifiedProp.Value.ToString(),
Action = action, ActionBy = GetUserName(),
ActionDate = DateTime.UtcNow.ToLongDateString(),
ObjectType = objectType
}).ToList();
return JsonConvert.SerializeObject(auditLog);
}

I think your best bet is to use JSON.NET to create two JSON objects, then recursively loop through the tree, comparing each node to see if it exists and is equal while you go.

I think the best way to go here is to create objects using newtonsoft json.http://www.nuget.org/packages/newtonsoft.json/
So, you will have two objects of the same type, which you can easily compare and mark the differences.

private IEnumerable<JProperty> JSONCompare(string expectedJSON, string actualJSON)
{
// convert JSON to object
JObject xptJson = JObject.Parse(expectedJSON);
JObject actualJson = JObject.Parse(actualJSON);
// read properties
var xptProps = xptJson.Properties().ToList();
var actProps = actualJson.Properties().ToList();
// find missing properties
var missingProps = xptProps.Where(expected => actProps.Where(actual => actual.Name == expected.Name).Count() == 0);
return missingProps;
}