We Keep Coding

How to deserialize JSON to IList<KeyValuePair<string, object>> with whitespaces in key?

I have a big problem with deserializing my JSON to an object. It should be deserialized to IList<KeyValuePair<string, object>> the problem is that the keys have white spaces.
"spec": {
"SOMETHING WITH SPACES" : "10"
etc.
...
}
public class SomeObject
{
...
public IList<KeyValuePair<string, object>> spec{ get; set; }
...
}
Deserializing code:
var sr = new ServiceStack.Text.JsonSerializer<SomeObject>();
var esResult = sr.DeserializeFromString(responseJson);
responseJson is a GET from ElasticSearch.
What I get to my field it is null.
If I have key without whitespaces it's deserializing normally and I'm getting my IList<KeyValuePair<string, object>>

You can't use IList or List here, because your source JSON has no [ ] in it, which is a requirement if you want to parse into such a collection. In other words, without [ ] you can't parse into a collection, at least not without going through lots of hoops.
Instead you need to use a Dictionary as was suggested already in comments.
Note: I used Newtonsoft JsonConvert because you didn't state what your parser is, but that should make little or no difference to my arguments.
Working code:
var json = "{ \"spec\": { \"SOMETHING WITH SPACES\" : \"10\" } }";
var someObj = JsonConvert.DeserializeObject<SomeObject>(json);
public class SomeObject
{
public Dictionary<string, object> spec{ get; set; }
}
After that, you can cast the spec property to an IEnumerable and loop through whatever was found:
foreach (var pair in someObj.spec as IEnumerable<KeyValuePair<string, object>>)
{
Console.WriteLine(pair.Key + " -> " + pair.Value);
}
Or even convert it to a List:
var list = someObj.spec.ToList();
foreach (var pair in list)
{
Console.WriteLine(pair.Key + " -> " + pair.Value);
}
.NET Fiddle: https://dotnetfiddle.net/15l2R3

If you don't mind using Newtonsoft.Json:
const string json = #"{""spec"": { ""SOMETHING WITH SPACES"" : ""10"", ""SOMETHING WITH MORE SPACES"" : ""20"" }}";
dynamic data = JsonConvert.DeserializeObject(json);
Dictionary<string, string> list = data["spec"].ToObject<Dictionary<string, string>>();
foreach (var item in list)
{
Console.WriteLine(item.Key + ", " + item.Value);
}

I guess your JSON serialiazer makes some trouble. I'd recommend to use Newtonsoft.Json (in NuGet)
I've tried following code, and it works fine:
var o1 = new SomeObject() { spec = new List<KeyValuePair<string, object>>() };
o1.spec.Add(new KeyValuePair<string, object>("test with spaces", 10));
var r1 = Newtonsoft.Json.JsonConvert.SerializeObject(o1);
Console.WriteLine(r1);
var o2 = Newtonsoft.Json.JsonConvert.DeserializeObject<SomeObject>(r1);
var r2 = Newtonsoft.Json.JsonConvert.SerializeObject(o2);
Console.WriteLine(r2);
The outcome is
{"spec":[{"Key":"test with spaces","Value":10}]}
{"spec":[{"Key":"test with spaces","Value":10}]}
No null values, all works fine.
EDIT: I actually see no reason, why spaces should be any problem at all. They are just part of the string.

Iterating JObject Keys

I have the following JObject as return by https://gate.io/api2#trade API. How do I iterate through each key which is a separate coin also get its value.
I tried to parse it using Newtonsoft JObject Parse like this:
var coinData = JObject.Parse(#"{
""result"": ""true"",
""available"": {
""BTC"": ""0.83337671"",
""LTC"": ""94.364"",
""ETH"": ""0.07161"",
""ETC"": ""82.35029899""
},
""locked"": {
""BTC"": ""0.0002"",
""YAC"": ""10.01""
}
}")["available"];
foreach (JToken item in coinData)
{
item.Key
}
but then JToken doesn't give access to key values. I don't know how to further parse it.
JSON received from gateio api:
{
"result": "true",
"available": {
"BTC": "0.83337671",
"LTC": "94.364",
"ETH": "0.07161",
"ETC": "82.35029899"
},
"locked": {
"BTC": "0.0002",
"YAC": "10.01"
}
}
EDIT: Should I break it with ':' while iterating in loop? This is working if i break it and replace quotes.
foreach (JToken item in coinData)
{
var data = item.ToString().Replace("\"", String.Empty).Split(':');
}
var data has two parts, 1 => coin name, 2 => balance.
Is there any other legit way?

JToken is base class for all types of json tokens. In your case though you want only json properties, so you need to filter by more narrow type - JProperty. You can filter to include only property tokens like this:
foreach (var item in coinData.OfType<JProperty>()) {
string coinName = item.Name;
// to parse as decimal
decimal balance = item.Value.Value<decimal>();
// or as string
string balanceAsString = item.Value.Value<string>();
}

I would suggest being very explicit about expecting the result of "available" to be another object, by casting to JObject. You can then call Properties() to get its properties, each as a JProperty. Here's a complete example to demonstrate:
using System;
using Newtonsoft.Json.Linq;
class Program
{
public static void Main()
{
string json = #"{
'result': 'true',
'available': {
'BTC': '0.83337671',
'LTC': '94.364',
'ETH': '0.07161',
'ETC': '82.35029899'
},
'locked': {
'BTC': '0.0002',
'YAC': '10.01'
}
}".Replace('\'', '"');
JObject root = JObject.Parse(json);
JObject coins = (JObject) root["available"];
foreach (JProperty property in coins.Properties())
{
string name = property.Name;
string value = (string) property.Value;
Console.WriteLine($"Name: {name}; Value: {value}");
}
}
}

Change the property names in a JSON object before conversion to XML

I am trying to convert some JSON to XML, but before that I need to change some properties to make a successful conversion.
Some of the properties in the JSON structure start with numbers, and when I try to make the conversion to XML I get an error because XML does not admit tags that start with numbers.
So, one solution that works for me is to change those property names that start with numbers by adding a prefix to the property.
I have been trying to do something like this:
public string ChangeNumericalPropertyNames(JsonReader reader)
{
JObject jo = JObject.Load(reader);
foreach (JProperty jp in jo.Properties())
{
if (Regex.IsMatch(jp.Name, #"^\d"))
{
string name = "n" + jp.Name;
//Logic to set changed name
}
}
return "Here I want to return the entire json string with changed names";
}
When I try this:
jp.Name = name;
Visual studio says that is not possible because jp.Name is read only.
Does anybody know how to achieve this solution?

Since the property name is read only, you'll need to replace the whole property. You can use the Replace method to do this:
if (Regex.IsMatch(jp.Name, #"^\d"))
{
string name = "n" + jp.Name;
jp.Replace(new JProperty(name, jp.Value));
}
However, this will lead to another problem-- since you are trying to modify the Properties collection of the JObject while iterating over it, Json.Net will throw an InvalidOperationException. To get around this, you must copy the the properties to a separate list and iterate over that instead. You can do that using the ToList() method in your foreach like this:
foreach (JProperty jp in jo.Properties().ToList())
Finally, to convert the updated JObject back to JSON, just use ToString(). Putting it all together we have:
public static string ChangeNumericalPropertyNames(JsonReader reader)
{
JObject jo = JObject.Load(reader);
foreach (JProperty jp in jo.Properties().ToList())
{
if (Regex.IsMatch(jp.Name, #"^\d"))
{
string name = "n" + jp.Name;
jp.Replace(new JProperty(name, jp.Value));
}
}
return jo.ToString();
}
Fiddle: https://dotnetfiddle.net/rX4Jyy
The above method will only handle a simple JSON object with properties all on one level. You indicated in your comment that your actual JSON is not flat, but hierarchical. In order to replace all of the numeric property names in a hierarchical structure, you'll need to make your method recursive, like this:
public static string ChangeNumericalPropertyNames(JsonReader reader)
{
JObject jo = JObject.Load(reader);
ChangeNumericalPropertyNames(jo);
return jo.ToString();
}
public static void ChangeNumericalPropertyNames(JObject jo)
{
foreach (JProperty jp in jo.Properties().ToList())
{
if (jp.Value.Type == JTokenType.Object)
{
ChangeNumericalPropertyNames((JObject)jp.Value);
}
else if (jp.Value.Type == JTokenType.Array)
{
foreach (JToken child in jp.Value)
{
if (child.Type == JTokenType.Object)
{
ChangeNumericalPropertyNames((JObject)child);
}
}
}
if (Regex.IsMatch(jp.Name, #"^\d"))
{
string name = "n" + jp.Name;
jp.Replace(new JProperty(name, jp.Value));
}
}
}
Fiddle: https://dotnetfiddle.net/qeZK1C

Serializing a list of dynamic objects to a CSV with ServiceStack.Text

All of my EF classes have a Projection() method that helps me choose what I want to project from the class to the SQL queries:
Example:
public static Expression<Func<Something, dynamic>> Projection()
{
return e => new
{
something = e.SomethingId,
name = e.Name,
requiredThingId = e.RequiredThingId,
requiredThing = new
{
requiredThingId = e.RequiredThing.RequiredThingId,
name = e.RequiredThing.Name,
...
},
optionalThingId = e.OptionalThingId,
optionalThing = e.OptionalThingId == null ? null : new
{
optionalThingId = e.OptionalThing.OptionalThingId,
name = e.OptionalThing.Name
}
...
};
}
I use these projection methods like this:
List<dynamic> projected = dbContext.Something.Select(Something.Projection()).ToList();
This way lets me reuse my projections all around my project.
I want to use ServiceStack.Text to serialize these lists to CSV.
I'm doing this:
string csvString = CsvSerializer.SerializeToCsv<dynamic>(Something.Select(Something.Projection()).ToList());
byte[] csvBytes = System.Text.Encoding.Unicode.GetBytes(csvString);
return File(csvBytes, "text/csv", "foo.csv");
But the result is an empty csv (csvString is full of "\r\n"'s and nothing more)
Questions:
Am I using correctly the SerializeToCsv() method?
Can I use <dynamic> in this library?
Suggestions for achieving my purpose?
For the example above the desired CSV would flatten all the properties, something like:
"somethingId";"name";"requiredThingId";"requiredThing.requiredThingId";"requiredThing.name";"optionalThingId";"optionalThing.optionalThingId";"optionalThing.name"

I will answer my own questions, but will not mark as accepted in hope of a new greater answer..
Am I using correctly the SerializeToCsv() method? Can I use dynamic
in this library?
Answer: Yes to both but ServiceStack.Text v4.0.36 is needed (which is available at MyGet, not Nuget)
Suggestions for achieving my purpose?
Answer: With ServiceStack.Text it is possible to serialize to CSV a List<dynamic>, but all the nested properties will be rendered as JSON and they will not be flattened out, eg:
List<dynamic> list = new List<dynamic>();
list.Add(new
{
name = "john",
pet = new
{
name = "doggy"
}
});
string csv = CsvSerializer.SerializeToCsv(list);
This list will be serialized to this csv:
name, pet
"john", { name = "doggy" }
And not to this csv, as I was expecting:
name, pet_name
"john", "doggy"
So... I finally ended up writing this code:
public class CsvHelper
{
public static string GetCSVString(List<dynamic> inputList)
{
var outputList = new List<Dictionary<string, object>>();
foreach (var item in inputList)
{
Dictionary<string, object> outputItem = new Dictionary<string, object>();
flatten(item, outputItem, "");
outputList.Add(outputItem);
}
List<string> headers = outputList.SelectMany(t => t.Keys).Distinct().ToList();
string csvString = ";" + string.Join(";", headers.ToArray()) + "\r\n";
foreach (var item in outputList)
{
foreach (string header in headers)
{
if (item.ContainsKey(header) && item[header] != null)
csvString = csvString + ";" + item[header].ToString();
else
csvString = csvString + ";";
}
csvString = csvString + "\r\n";
}
return csvString;
}
private static void flatten(dynamic item, Dictionary<string, object> outputItem, string prefix)
{
if (item == null)
return;
foreach (PropertyInfo propertyInfo in item.GetType().GetProperties())
{
if (!propertyInfo.PropertyType.Name.Contains("AnonymousType"))
outputItem.Add(prefix + "__" + propertyInfo.Name, propertyInfo.GetValue(item));
else
flatten(propertyInfo.GetValue(item), outputItem, (prefix.Equals("") ? propertyInfo.Name : prefix + "__" + propertyInfo.Name));
}
}
}
What this does is:
It flattens the List, so that all the properties of the objects in the list are primitives (eg: no nested properties)
It creates a CSV from that flattened list.
This algorithm is O(n*m), being
n: number of items in the list
m: number of properties inside each item (including nested properties)

Whilst the recommendation is to use clean POCO's for Serialization, you can serialize in-line anonymous types, e.g:
var somethings = dbContext.Something.Select(e => new {
something = e.SomethingId,
name = e.Name
});
somethings.ToCsv().Print();
Otherwise I've just added support for serializing IEnumerable<object> and IEnumerable<dynamic> in this commit.
This change is available from v4.0.36+ that's now available on MyGet.

I believe that part of what is going on is that the library is using properties in the object to determine what to serialize. I believe that a dynamic object does not construct properties like it's a POCO. If you don't setup a getter and setter on your object, it certainly won't work. Just for reference, I using version 4.5.6.0 of this library.

Iterate through dynamic form object

Using mvc i get values like this to avoid class declarations and router changes.
public dynamic Create([FromBody] dynamic form)
{
var username = form["username"].Value;
var password = form["password"].Value;
var firstname = form["firstname"].Value;
...
I like to iterate through all values and check them for null or empty.

If you get a json from the argument, you could convert it to an Dictionary<string, dynamic> where the string key is the name of the property and the dynamic is a value that can assume any type. For sample:
var d = JsonConvert.DeserializeObject<Dictionary<string, dynamic>>(form);
var username = d["username"];
You also could loop between Keys property from the Dictionary<>:
foreach(var key in d.Keys)
{
// check if the value is not null or empty.
if (!string.IsNullOrEmpty(d[key]))
{
var value = d[key];
// code to do something with
}
}

This is quite old, but I came across this and am wondering why the following was not proposed:
var data = (IDictionary<string, object>)form;

You can use JavaScriptSerializer and dynamic object:
JavaScriptSerializer serializer = new JavaScriptSerializer();
dynamic myDynamicObject = serializer.DeserializeObject(json);
For example, if you want to loop through myDynamicObject["users"]:
foreach (KeyValuePair<string, dynamic> user in myDynamicObject["users"]){
Console.WriteLine(user.Key+": "+user.Value["username"]);
Console.WriteLine(user.Key+": "+user.Value["email"]);
}