We Keep Coding

Accepting raw JSON Asp.net core

I have a asp.net core method, I want it to accept RAW json, I do not and will not always know the schema of the json object, so I am using dynamic types with dot notation.
This method works when I string the json escaping each character. I have tried to use the json body directly, but this did not work. So it seems my option were to Serialize and then Deserialize the json. ( very redundant) but it seems to throw error any other way if I try to use the JSON body directly.
In the debugger, everything seems to work with the Serialize and Deserialize of the object / string, but throws an error on the id(property) when I try to cast the object to string and gives the error. (In the debugger I am able to see the Id correctly though).
({Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: Cannot convert type 'System.Text.Json.JsonElement' to 'string')}
I really do not see why it gives the type as a string yet cannot convert it. I have even tried to remove the casting, and still receive this error.
public IActionResult Post([FromBody] ExpandoObject requestInput)
{
try
{
//Makes a JSON String
var stringObject = (string) JsonSerializer.Serialize(requestInput);
DateTime time = DateTime.UtcNow;
// Recreated the Json Object
dynamic requestObject = JsonSerializer.Deserialize<ExpandoObject>(stringObject);
// Throws Error here, yet it shows Value as the correct Id number (Value: Type String)
string reqObject = (string) requestObject.Id;

So there is no support for ExpandoObject in .NET Core, yet. MS says that maybe it will be added in .NET 5.0. Until then, you can use this JsonConverter I found on a thread. I will post the code here in case that thread goes away.
You can use it like this:
[HttpPost, Route("testPost")]
public IActionResult TestPost([FromBody] object obj) // just use "object"
{
// object is: { "hello":"world" }
var myDynamic = JsonSerializer.Deserialize<dynamic>(
JsonSerializer.Serialize(obj), new JsonSerializerOptions
{
Converters = { new DynamicJsonConverter() }
});
var test = (string)myDynamic.hello;
// test will equal "world"
return Ok();
}
Here is the converter:
/// <summary>
/// Temp Dynamic Converter
/// by:tchivs#live.cn
/// </summary>
public class DynamicJsonConverter : JsonConverter<dynamic>
{
public override dynamic Read(ref Utf8JsonReader reader,
Type typeToConvert,
JsonSerializerOptions options)
{
if (reader.TokenType == JsonTokenType.True)
{
return true;
}
if (reader.TokenType == JsonTokenType.False)
{
return false;
}
if (reader.TokenType == JsonTokenType.Number)
{
if (reader.TryGetInt64(out long l))
{
return l;
}
return reader.GetDouble();
}
if (reader.TokenType == JsonTokenType.String)
{
if (reader.TryGetDateTime(out DateTime datetime))
{
return datetime;
}
return reader.GetString();
}
if (reader.TokenType == JsonTokenType.StartObject)
{
using JsonDocument documentV = JsonDocument.ParseValue(ref reader);
return ReadObject(documentV.RootElement);
}
// Use JsonElement as fallback.
// Newtonsoft uses JArray or JObject.
JsonDocument document = JsonDocument.ParseValue(ref reader);
return document.RootElement.Clone();
}
private object ReadObject(JsonElement jsonElement)
{
IDictionary<string, object> expandoObject = new ExpandoObject();
foreach (var obj in jsonElement.EnumerateObject())
{
var k = obj.Name;
var value = ReadValue(obj.Value);
expandoObject[k] = value;
}
return expandoObject;
}
private object? ReadValue(JsonElement jsonElement)
{
object? result = null;
switch (jsonElement.ValueKind)
{
case JsonValueKind.Object:
result = ReadObject(jsonElement);
break;
case JsonValueKind.Array:
result = ReadList(jsonElement);
break;
case JsonValueKind.String:
//TODO: Missing Datetime&Bytes Convert
result = jsonElement.GetString();
break;
case JsonValueKind.Number:
//TODO: more num type
result = 0;
if (jsonElement.TryGetInt64(out long l))
{
result = l;
}
break;
case JsonValueKind.True:
result = true;
break;
case JsonValueKind.False:
result = false;
break;
case JsonValueKind.Undefined:
case JsonValueKind.Null:
result = null;
break;
default:
throw new ArgumentOutOfRangeException();
}
return result;
}
private object? ReadList(JsonElement jsonElement)
{
IList<object?> list = new List<object?>();
foreach (var item in jsonElement.EnumerateArray())
{
list.Add(ReadValue(item));
}
return list.Count == 0 ? null : list;
}
public override void Write(Utf8JsonWriter writer,
object value,
JsonSerializerOptions options)
{
// writer.WriteStringValue(value.ToString());
}
}
Edited To Add:
Here is a much slicker way to handle dynamic using the converter above as pointed out by Aluan in the comments. In your Startup.cs class, add this:
services.AddControllers().AddJsonOptions(options =>
options.JsonSerializerOptions.Converters.Add(new DynamicJsonConverter()));
Then you don't have to do any goofy stuff in your controller. You can just set the body parameter as dynamic and it magically works:
[HttpPost, Route("testPost")]
public IActionResult TestPost([FromBody] dynamic obj)
{
// object is: { "hello":"world" }
var test = (string)obj.hello;
// test will equal "world"
return Ok();
}
Way nicer!

Cannot Deserialize the Current JSON Object (Empty Array)

I am trying to make this program that formats all these objects into a treeview, to do this (I'm using JSON for ordering the objects), I needed to parse the JSON, so I chose JSON.NET.
So here is an example of how the formatting is:
{
"Space": {
"ClassName": "SpaceObject",
"Name": "Space",
"Children": {
"Object1": {
"ClassName": "Object",
"Name": "Object1",
"Children": []
},
"Object2": {
"ClassName": "Object",
"Name": "Object2",
"Children": []
}
}
}
}
public class CObject
{
[JsonProperty(PropertyName = "Name")]
public string Name { get; set; }
[JsonProperty(PropertyName = "ClassName")]
public string ClassName { get; set; }
[JsonProperty(PropertyName = "Children")]
public IDictionary<string, CObject> Children { get; set; }
}
IDictionary<string, CObject> obj = JsonConvert.DeserializeObject<IDictionary<string, CObject>>(Json, new JsonSerializerSettings() {
MissingMemberHandling = MissingMemberHandling.Ignore,
NullValueHandling = NullValueHandling.Ignore,
});
foreach (var i in obj) {
ExplorerView1.Nodes.Add(AddObject(i.Value));
}
I believe I found the error, it's due to a children array having no objects in it. I don't know how to fix this though, can anyone help?

JsonSingleOrEmptyArrayConverter<T> from this answer to Deserialize JSON when type can be different almost does what you need. It simply needs to be enhanced to allow the current contract type to be a dictionary contract as well as an object contract.
First, modify JsonSingleOrEmptyArrayConverter<T> as follows:
public class JsonSingleOrEmptyArrayConverter<T> : JsonConverter where T : class
{
//https://stackoverflow.com/questions/29449641/deserialize-json-when-type-can-be-different?rq=1
public override bool CanConvert(Type objectType)
{
return typeof(T).IsAssignableFrom(objectType);
}
public override bool CanWrite { get { return false; } }
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var contract = serializer.ContractResolver.ResolveContract(objectType);
// Allow for dictionary contracts as well as objects contracts, since both are represented by
// an unordered set of name/value pairs that begins with { (left brace) and ends with } (right brace).
if (!(contract is Newtonsoft.Json.Serialization.JsonObjectContract
|| contract is Newtonsoft.Json.Serialization.JsonDictionaryContract))
{
throw new JsonSerializationException(string.Format("Unsupported objectType {0} at {1}.", objectType, reader.Path));
}
switch (reader.SkipComments().TokenType)
{
case JsonToken.StartArray:
{
int count = 0;
while (reader.Read())
{
switch (reader.TokenType)
{
case JsonToken.Comment:
break;
case JsonToken.EndArray:
// You might want to allocate an empty object here if existingValue is null
// If so, do
// return existingValue ?? contract.DefaultCreator();
return existingValue;
default:
{
count++;
if (count > 1)
throw new JsonSerializationException(string.Format("Too many objects at path {0}.", reader.Path));
existingValue = existingValue ?? contract.DefaultCreator();
serializer.Populate(reader, existingValue);
}
break;
}
}
// Should not come here.
throw new JsonSerializationException(string.Format("Unclosed array at path {0}.", reader.Path));
}
case JsonToken.Null:
return null;
case JsonToken.StartObject:
existingValue = existingValue ?? contract.DefaultCreator();
serializer.Populate(reader, existingValue);
return existingValue;
default:
throw new InvalidOperationException("Unexpected token type " + reader.TokenType.ToString());
}
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
public static partial class JsonExtensions
{
public static JsonReader SkipComments(this JsonReader reader)
{
while (reader.TokenType == JsonToken.Comment && reader.Read())
;
return reader;
}
}
Then deserialize as follows:
var settings = new JsonSerializerSettings
{
MissingMemberHandling = MissingMemberHandling.Ignore,
NullValueHandling = NullValueHandling.Ignore,
Converters = { new JsonSingleOrEmptyArrayConverter<IDictionary<string, CObject>>() },
};
var obj = JsonConvert.DeserializeObject<IDictionary<string, CObject>>(Json, settings);
Notes:
My assumption is that an empty array [] is only used when there are no children. If the array is ever nonempty, this assumption will be wrong and the converter will not work correctly.
The converter returns a null value for an empty array. If you would instead prefer an empty dictionary, uncomment the following lines:
// You might want to allocate an empty object here if existingValue is null
// If so, do
// return existingValue ?? contract.DefaultCreator();
Working .Net fiddle here.

How to ignore specific dictionary key in Json.NET deserialization?

How can one deserialize the following JSON
{
"result" : {
"master" : [
["one", "two"],
["three", "four"],
["five", "six", "seven"],
],
"blaster" : [
["ein", "zwei"],
["drei", "vier"]
],
"surprise" : "nonsense-nonsense-nonsense"
}
}
into the following data structure
class ResultView
{
public Dictionary<string, string[][]> Result { get; set; }
}
with Json.NET?
It has to be dictionary because key names such as 'master' and 'blaster' are unknown at the time of compilation. What is known is that they always point to an array of arrays of strings. The problem is that key 'surprise', whose name is known and always the same, points to something that cannot be interpreted as string[][], and this leads to exception in Json.NET.
Is there any way to make Json.NET ignore specific dictionary key?

You can introduce a custom generic JsonConverter for IDictionary<string, TValue> that filters out invalid dictionary values (i.e. those that cannot be deserialized successfully to the dictionary value type):
public class TolerantDictionaryItemConverter<TDictionary, TValue> : JsonConverter where TDictionary : IDictionary<string, TValue>
{
public override bool CanConvert(Type objectType)
{
return typeof(TDictionary).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type dictionaryType, object existingValue, JsonSerializer serializer)
{
// Get contract information
var contract = serializer.ContractResolver.ResolveContract(dictionaryType) as JsonDictionaryContract;
if (contract == null)
throw new JsonSerializationException(string.Format("Invalid JsonDictionaryContract for {0}", dictionaryType));
if (contract.DictionaryKeyType != typeof(string))
throw new JsonSerializationException(string.Format("Key type {0} not supported", dictionaryType));
var itemContract = serializer.ContractResolver.ResolveContract(contract.DictionaryValueType);
// Process the first token
var tokenType = reader.SkipComments().TokenType;
if (tokenType == JsonToken.Null)
return null;
if (reader.TokenType != JsonToken.StartObject)
throw new JsonSerializationException(string.Format("Expected {0}, encountered {1} at path {2}", JsonToken.StartArray, reader.TokenType, reader.Path));
// Allocate the dictionary
var dictionary = existingValue as IDictionary<string, TValue> ?? (IDictionary<string, TValue>) contract.DefaultCreator();
// Process the collection items
while (reader.Read())
{
if (reader.TokenType == JsonToken.EndObject)
{
return dictionary;
}
else if (reader.TokenType == JsonToken.PropertyName)
{
var key = (string)reader.Value;
reader.ReadSkipCommentsAndAssert();
// For performance, skip tokens we can easily determine cannot be deserialized to itemContract
if (itemContract.QuickRejectStartToken(reader.TokenType))
{
System.Diagnostics.Debug.WriteLine(string.Format("value for {0} skipped", key));
reader.Skip();
}
else
{
// What we want to do is to distinguish between JSON files that are not WELL-FORMED
// (e.g. truncated) and that are not VALID (cannot be deserialized to the current item type).
// An exception must still be thrown for an ill-formed file.
// Thus we first load into a JToken, then deserialize.
var token = JToken.Load(reader);
try
{
var value = serializer.Deserialize<TValue>(token.CreateReader());
dictionary.Add(key, value);
}
catch (Exception)
{
System.Diagnostics.Debug.WriteLine(string.Format("value for {0} skipped", key));
}
}
}
else if (reader.TokenType == JsonToken.Comment)
{
continue;
}
else
{
throw new JsonSerializationException(string.Format("Unexpected token type {0} object at path {1}.", reader.TokenType, reader.Path));
}
}
// Should not come here.
throw new JsonSerializationException("Unclosed object at path: " + reader.Path);
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
public static partial class JsonExtensions
{
public static JsonReader SkipComments(this JsonReader reader)
{
while (reader.TokenType == JsonToken.Comment && reader.Read())
;
return reader;
}
public static void ReadSkipCommentsAndAssert(this JsonReader reader)
{
if (reader == null)
throw new ArgumentNullException();
while (reader.Read())
{
if (reader.TokenType != JsonToken.Comment)
return;
}
new JsonReaderException(string.Format("Unexpected end at path {0}", reader.Path));
}
internal static bool QuickRejectStartToken(this JsonContract contract, JsonToken token)
{
if (contract is JsonLinqContract)
return false;
switch (token)
{
case JsonToken.None:
return true;
case JsonToken.StartObject:
return !(contract is JsonContainerContract) || contract is JsonArrayContract; // reject if not dictionary or object
case JsonToken.StartArray:
return !(contract is JsonArrayContract); // reject if not array
case JsonToken.Null:
return contract.CreatedType.IsValueType && Nullable.GetUnderlyingType(contract.UnderlyingType) == null;
// Primitives
case JsonToken.Integer:
case JsonToken.Float:
case JsonToken.String:
case JsonToken.Boolean:
case JsonToken.Undefined:
case JsonToken.Date:
case JsonToken.Bytes:
return !(contract is JsonPrimitiveContract); // reject if not primitive.
default:
return false;
}
}
}
Then you can add it to settings as follows:
var settings = new JsonSerializerSettings
{
Converters = { new TolerantDictionaryItemConverter<IDictionary<string, TValue>, TValue>() },
};
var root = JsonConvert.DeserializeObject<ResultView>(json, settings);
Or add it directly to ResultView with JsonConverterAttribute:
class ResultView
{
[JsonConverter(typeof(TolerantDictionaryItemConverter<IDictionary<string, string[][]>, string[][]>))]
public Dictionary<string, string[][]> Result { get; set; }
}
Notes:
I wrote the converter in a general way to handle any type of dictionary value including primitives such as int or DateTime as well as arrays or objects.
While a JSON file with invalid dictionary values (ones that cannot be deserialized to the dictionary value type) should be deserializable, an ill-formed JSON file (e.g. one that is truncated) should still result in an exception being thrown.
The converter handles this by first loading the value into a JToken then attempting to deserialize the token. If the file is ill-formed, JToken.Load(reader) will throw an exception, which is intentionally not caught.
Json.NET's exception handling is reported to be "very flaky" (see e.g. Issue #1580: Regression from Json.NET v6: cannot skip an invalid object value type in an array via exception handling) so I did not rely on it to skip invalid dictionary values.
I'm not 100% sure I got all cases of comment handling correct. So that may need additional testing.
Working sample .Net fiddle here.

I think you could ignore exceptions like this:
ResultView result = JsonConvert.DeserializeObject<ResultView>(jsonString,
new JsonSerializerSettings
{
Error = delegate (object sender, Newtonsoft.Json.Serialization.ErrorEventArgs args)
{
// System.Diagnostics.Debug.WriteLine(args.ErrorContext.Error.Message);
args.ErrorContext.Handled = true;
}
}
);
args.ErrorContext.Error.Message would contain the actual error message.
args.ErrorContext.Handled = true; will tell Json.Net to proceed.

JsonSerializer.CreateDefault().Populate(..) resets my values

I have following class:
public class MainClass
{
public static MainClass[] array = new MainClass[1]
{
new MainClass
{
subClass = new SubClass[2]
{
new SubClass
{
variable1 = "my value"
},
new SubClass
{
variable1 = "my value"
}
}
}
};
public SubClass[] subClass;
[DataContract]
public class SubClass
{
public string variable1 = "default value";
[DataMember] // because only variable2 should be saved in json
public string variable2 = "default value";
}
}
which I save as follows:
File.WriteAllText("data.txt", JsonConvert.SerializeObject(new
{
MainClass.array
}, new JsonSerializerSettings { Formatting = Formatting.Indented }));
data.txt:
{
"array": [
{
"subClass": [
{
"variable2": "value from json"
},
{
"variable2": "value from json"
}
]
}
]
}
then I deserialize and populate my object like this:
JObject json = JObject.Parse(File.ReadAllText("data.txt"));
if (json["array"] != null)
{
for (int i = 0, len = json["array"].Count(); i < len; i++)
{
using (var sr = json["array"][i].CreateReader())
{
JsonSerializer.CreateDefault().Populate(sr, MainClass.array[i]);
}
}
}
however, when I print following variables:
Console.WriteLine(MainClass.array[0].subClass[0].variable1);
Console.WriteLine(MainClass.array[0].subClass[0].variable2);
Console.WriteLine(MainClass.array[0].subClass[1].variable1);
Console.WriteLine(MainClass.array[0].subClass[1].variable2);
then output of it is:
default value
value from json
default value
value from json
but instead of "default value" there should be "my value" because that is what I used while creating an instance of class and JsonSerializer should only populate the object with values from json.
How do I properly populate the whole object without resetting its properties which are not included in json?

It looks as though JsonSerializer.Populate() lacks the MergeArrayHandling setting that is available for JObject.Merge(). Through testing I have found that:
Populating members that are arrays or some other type of read-only collection seems to work like MergeArrayHandling.Replace.
This is the behavior you are experiencing -- the existing array and all the items therein are being discarded and replaced with a fresh array containing newly constructed items that have default values. In contrast, you require MergeArrayHandling.Merge: Merge array items together, matched by index.
Populating members that are read/write collections such as List<T> seems to work like MergeArrayHandling.Concat.
It seems reasonable to request an enhancement that Populate() support this setting -- though I don't know how easy it would be to implement. At the minimum the documentation for Populate() should explain this behavior.
In the meantime, here's a custom JsonConverter with the necessary logic to emulate the behavior of MergeArrayHandling.Merge:
public class ArrayMergeConverter<T> : ArrayMergeConverter
{
public override bool CanConvert(Type objectType)
{
return objectType.IsArray && objectType.GetArrayRank() == 1 && objectType.GetElementType() == typeof(T);
}
}
public class ArrayMergeConverter : JsonConverter
{
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (!objectType.IsArray)
throw new JsonSerializationException(string.Format("Non-array type {0} not supported.", objectType));
var contract = (JsonArrayContract)serializer.ContractResolver.ResolveContract(objectType);
if (contract.IsMultidimensionalArray)
throw new JsonSerializationException("Multidimensional arrays not supported.");
if (reader.TokenType == JsonToken.Null)
return null;
if (reader.TokenType != JsonToken.StartArray)
throw new JsonSerializationException(string.Format("Invalid start token: {0}", reader.TokenType));
var itemType = contract.CollectionItemType;
var existingList = existingValue as IList;
IList list = new List<object>();
while (reader.Read())
{
switch (reader.TokenType)
{
case JsonToken.Comment:
break;
case JsonToken.Null:
list.Add(null);
break;
case JsonToken.EndArray:
var array = Array.CreateInstance(itemType, list.Count);
list.CopyTo(array, 0);
return array;
default:
// Add item to list
var existingItem = existingList != null && list.Count < existingList.Count ? existingList[list.Count] : null;
if (existingItem == null)
{
existingItem = serializer.Deserialize(reader, itemType);
}
else
{
serializer.Populate(reader, existingItem);
}
list.Add(existingItem);
break;
}
}
// Should not come here.
throw new JsonSerializationException("Unclosed array at path: " + reader.Path);
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override bool CanConvert(Type objectType)
{
throw new NotImplementedException();
}
}
Then add the converter to your subClass member as follows:
[JsonConverter(typeof(ArrayMergeConverter))]
public SubClass[] subClass;
Or, if you don't want to add Json.NET attributes to your data model, you can add it in serializer settings:
var settings = new JsonSerializerSettings
{
Converters = new[] { new ArrayMergeConverter<MainClass.SubClass>() },
};
JsonSerializer.CreateDefault(settings).Populate(sr, MainClass.array[i]);
The converter is specifically designed for arrays but a similar converter could easily be created for read/write collections such as List<T>.

Prevent Json.NET from deserializing arrays of strings as arrays of key-value pairs

JsonConvert.DeserializeObject successfully deserializes ['a','b'] as List<KeyValuePair<string, object>>. I would like it to fail, only succeeding when the input string is like [{'Key':'a','Value':'b'}].
Is there a way to accomplish this?

It appears you may have found a bug in Json.NET's KeyValuePairConverter, namely that it assumes the reader is positioned at the beginning of a JSON object rather than checking and validating that it is. You could report an issue on it if you want.
In the meantime, the following JsonConverter will correctly throw a JsonException for your case:
public class KeyValueConverter : JsonConverter
{
interface IToKeyValuePair
{
object ToKeyValuePair();
}
struct Pair<TKey, TValue> : IToKeyValuePair
{
public TKey Key { get; set; }
public TValue Value { get; set; }
public object ToKeyValuePair()
{
return new KeyValuePair<TKey, TValue>(Key, Value);
}
}
public override bool CanConvert(Type objectType)
{
bool isNullable = (Nullable.GetUnderlyingType(objectType) != null);
Type type = (Nullable.GetUnderlyingType(objectType) ?? objectType);
return type.IsGenericType
&& type.GetGenericTypeDefinition() == typeof(KeyValuePair<,>);
}
public override bool CanWrite { get { return false; } } // Use Json.NET's writer.
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
bool isNullable = (Nullable.GetUnderlyingType(objectType) != null);
Type type = (Nullable.GetUnderlyingType(objectType) ?? objectType);
if (isNullable && reader.TokenType == JsonToken.Null)
return null;
if (type.IsGenericType
&& type.GetGenericTypeDefinition() == typeof(KeyValuePair<,>))
{
var pairType = typeof(Pair<,>).MakeGenericType(type.GetGenericArguments());
var pair = serializer.Deserialize(reader, pairType);
if (pair == null)
return null;
return ((IToKeyValuePair)pair).ToKeyValuePair();
}
else
{
throw new JsonSerializationException("Invalid type: " + objectType);
}
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Then use it like:
string json = #"['a','b']";
var settings = new JsonSerializerSettings { Converters = new JsonConverter[] { new KeyValueConverter() } };
var list = JsonConvert.DeserializeObject<List<KeyValuePair<string, object>>>(json, settings);
Example fiddle.
Update
To force an error when the JSON contains a non-existent property, use JsonSerializerSettings.MissingMemberHandling = MissingMemberHandling.Error.
var settings = new JsonSerializerSettings
{
MissingMemberHandling = MissingMemberHandling.Error,
Converters = new JsonConverter[] { new KeyValueConverter() },
};