We Keep Coding

How to automatically deserialize a JSON property value when the value is a double-serialized escaped JSON string?

Let's say I have the following type:
class Foo<T>{
public T data {get;init;}
}
and I want to use this type in multiple controllers:
[HttpPost]
public async void Post(Foo<SomeComplexClass> myFoo){
myFoo.data // this is SomeComplexClass
// Dostuff
}
[HttpPost]
public async void Post2(Foo<OtherMoreDifferentClass>foo2){
foo2.data // this is OtherMoreDifferentClass
}
This is all fine and good, except that the caller of this api which I have no control over serializes T into json before sending the http request, so everything is coming in as Foo<string>. I'm looking into writing a converter for Foo that's able to take this string and automatically convert it to the requested type by deserializing it to the requested type in the JsonConverter.
I'd like to not have to re-write my controllers or put serialization code in my controllers either
The problem is that the JsonConverter<T> class requires that you implement read and write using a utf8jsonreader and writer, which are the lowest level apis in the System.Text.Json library. These deal with individual tokens! All I want to do is take the full string that's coming over the wire, serialize it into a Foo<string> then taking foo.data and re-deserialize it into the requested type if that makes any sense.
Is there a way I can trade this utf8jsonreader for one of the higher level api's somehow? Or is this converter business the wrong approach for this problem?
Update
To clarify, say I have the following class:
public class SomeComplexClass
{
public string Foo { get; set; } = "";
}
Under normal circumstances, it should be serialized as follows:
{"data":{"Foo":"hello"}}
But the caller of my API is sending the following:
{"data":"{\"Foo\":\"hello\"}"}
As you can seem the value of data has been double-serialized: once to a string, then a second time when the container object is serialized over the wire. I would like to have some automatic way of binding the embedded double-serialized JSON to my Foo<T>.data value.

You can create a custom JsonConverter that, in Read(), checks to see whether the current JSON token is a string, and if so, deserializes the string as a string, then deserializes that string to the final data model T. The converter should be applied directly to Foo.data:
class Foo<T>
{
[JsonConverter(typeof(DoubleSerializedConverter))]
public T data {get; init;}
}
public class DoubleSerializedConverter : JsonConverterFactory
{
public override bool CanConvert(Type typeToConvert) => true;
class ObjectOrArrayKindConverter<T> : JsonConverter<T>
{
public override T? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) =>
reader.TokenType switch
{
JsonTokenType.String => JsonSerializer.Deserialize<T>(reader.GetString()!, options),
_ => JsonSerializer.Deserialize<T>(ref reader, options),
};
public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options) => JsonSerializer.Serialize(writer, value, options);
}
class UnknownKindConverter<T> : JsonConverter<T>
{
// TODO: decide how to handle primitive types like string, byte [] and so on. There's no foolproof way to check whether they were conditionally double serialized,
// so you must either assume they were always double-serialized, or assume they were never double-serialized.
//public override T? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) => JsonSerializer.Deserialize<T>(ref reader, options);
public override T? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) =>
reader.TokenType switch
{
JsonTokenType.String => JsonSerializer.Deserialize<T>(reader.GetString()!, options), // Use if you want to assume strings and other primitives are double serialized.
//JsonTokenType.String => JsonSerializer.Deserialize<T>(ref reader, options), // Use if you want to assume strings and other primitives are not double serialized.
_ => JsonSerializer.Deserialize<T>(ref reader, options),
};
public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options) => JsonSerializer.Serialize(writer, value, options);
}
public sealed override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
{
var typeInfo = JsonSerializerOptions.Default.GetTypeInfo(typeToConvert);
var genericConverterType = (typeInfo.Kind == JsonTypeInfoKind.None ? typeof(UnknownKindConverter<>) : typeof(ObjectOrArrayKindConverter<>));
return (JsonConverter)Activator.CreateInstance(genericConverterType.MakeGenericType(typeToConvert))!;
}
}
Notes:
It is possible to reliably check whether a POCO that is normally serialized as a JSON object or array was double-serialized, but it is impossible to reliably check whether a .NET primitive was double serialized. E.g. the raw JSON string "\"hello\"" might be a doubly-serialized hello or a singly-serialized "hello".
Thus you will need to decide what to do for primitives. The converter above assumes primitives are double-serialized, but if you would prefer the opposite, in UnknownKindConverter<T> above uncomment
//JsonTokenType.String => JsonSerializer.Deserialize<T>(ref reader, options)`
JsonSerializerOptions.GetTypeInfo(Type) is used to determine whether the T data is normally serialized as an JSON object or array. This API was introduced in .NET 7 so if you are working in an earlier version you may need to replace that logic with something handcrafted.
Demo fiddle here

System.Text.Json: How to convert from "true" to boolean

If I receive JSON which is beyond my control, which has a property as follows.
{"allow":"true"}
and I want that to map to a bool property in C#.
I saw that I can do similar for numbers using the attribute
[JsonNumberHandling(JsonNumberHandling.AllowReadingFromString)]
but how can I do that for booleans?
Edit:
I am not sure why this is closed because of a duplicate question, since the other question focuses on converting from ints to bools

Ok, it seems like this functionality is not built in but is easy to implement.
https://github.com/dotnet/runtime/issues/43587#issuecomment-780031498
Would have to create a custom converter as follows (taken from the link).
public class BooleanConverter : JsonConverter<bool>
{
public override bool Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
switch (reader.TokenType)
{
case JsonTokenType.True:
return true;
case JsonTokenType.False:
return false;
case JsonTokenType.String:
return reader.GetString() switch
{
"true" => true,
"false" => false,
_ => throw new JsonException()
};
default:
throw new JsonException();
}
}
public override void Write(Utf8JsonWriter writer, bool value, JsonSerializerOptions options)
{
writer.WriteBooleanValue(value);
}
}

Just for a record, any converters take a time and memory. So if you have not very many string bool properies in your json string, the easiest and the most sufficient way would be to fix the json string
var json="{\"allow\":\"true\"}";
json=json.Replace("\"true\"","true").Replace("\"false\"","false");
output
{"allow":true}

C# Asp.net Json CustomConverter error when converting from string to object, when invoked from API call

I use C# ASP.NET Net6 to create a webapi. There I use a custom jsonconverter to convert a custom class which is called Password. The string comes from the body of a request dto. During converting from string to object I check with a regex if the password meets requirements and throw an exception in case of failure. This causes my application to break.
What is the proper way to check the string, which is to be parsed, before converting to an object?
Do I need to throw a special exception?
Can I somehow check the value in CanConvert before Read Function?
namespace TestApp.Converters
{
public class PasswordJsonConverter : JsonConverter<Password>
{
public override bool CanConvert(Type typeToConvert)
{
return typeToConvert == typeof(Password);
}
public override Password? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
if(!Password.TryParse(reader.GetString(), out Password pw))
{
throw new JsonException("test"); // <= this causes the application to stop
}
return pw;
}
public override void Write(Utf8JsonWriter writer, Password value, JsonSerializerOptions options)
{
writer.WriteStringValue(value.ToString());
}
}
}

How can I serialize only the contents of an object property using System.Text.Json (custom converter or other method)

I am developing a client for an API which expects non conventional json and I cannot find a way produce what it wants using System.Text.Json.
The api in question expects json in the following format:
{
"user__0__id": XXXX,
"user__0__email": "XXXX",
"user__1__id": YYYY,
"user__1__email": "YYYY"
}
I have classes as such:
class Request {
public List<User> Users {get; set;}
}
class User {
public int Id {get; set;}
public string Email {get; set; }
}
I have tried using a custom json converter with a write method like this:
public override void Write(Utf8JsonWriter writer, List<User> value, JsonSerializerOptions options)
{
for (var x = 0; x < value.Count; x++)
{
var user = value[x];
writer.WriteString($"users__{x}__id", user.Id.ToString());
writer.WriteString($"users__{x}__email", user.Email);
}
}
This kind of works, but the problem is that the serializer will first output the property name ("Users:") and then the rest.
Eg
{
"Users":"user__0__id":XXXX,
"user__0__email":"XXXX",
"user__1__id":YYYY
"user__1__email":"YYYY"
}
Obviously the above doesn't work as it's invalid json.
I've researched docs but couldn't find a way to skip writing the property name and only write the contents of the property, as that's what I need to do basically.

You can generate the required JSON from your Request class using this JsonConverter<Request> rather than a JsonConverter<List<User>> as you are currently attempting:
class RequestConverter : JsonConverter<Request>
{
public override void Write(Utf8JsonWriter writer, Request value, JsonSerializerOptions options)
{
writer.WriteStartObject();
for (int x = 0, n = (value.Users == null ? 0 : value.Users.Count); x < n; x++)
{
var user = value.Users[x];
writer.WriteNumber($"users__{x.ToString(NumberFormatInfo.InvariantInfo)}__id", user.Id);
writer.WriteString($"users__{x.ToString(NumberFormatInfo.InvariantInfo)}__email", user.Email);
}
writer.WriteEndObject();
}
public override Request Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) => throw new NotImplementedException();
}
If you prefer to separate the responsibilities for serializing Request and List<User>, you will need converters for both:
class RequestConverter : JsonConverter<Request>
{
public override void Write(Utf8JsonWriter writer, Request value, JsonSerializerOptions options) =>
JsonSerializer.Serialize(writer, value.Users, options);
public override Request Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) =>
new Request { Users = JsonSerializer.Deserialize<List<User>>(ref reader, options) };
}
class UserListConverter : JsonConverter<List<User>>
{
public override void Write(Utf8JsonWriter writer, List<User> value, JsonSerializerOptions options)
{
writer.WriteStartObject();
for (int x = 0, n = value.Count; x < n; x++)
{
var user = value[x];
writer.WriteNumber($"users__{x.ToString(NumberFormatInfo.InvariantInfo)}__id", user.Id);
writer.WriteString($"users__{x.ToString(NumberFormatInfo.InvariantInfo)}__email", user.Email);
}
writer.WriteEndObject();
}
public override List<User> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) => throw new NotImplementedException();
}
Notes:
The reason the extra "Users:" property shows up is that you created your converter for the inner List<Users> property. Thus JsonSerializer will serialize the outer Request object using default serialization, then use the converter for the value of Users, which results an outer wrapper object like so:
{ "Users": /* Value of Users */ }
To avoid the wrapper object you need to either serialize List<User> directly, or apply the converter to Request instead, which is what I do above.
In a contract-based serializer such as System.Text.Json (or any other contract-based serializer as far as I know) a child class cannot "reach up" and control the serialization of a parent class. It's the responsibility of each class (or its applicable converter) to define a contract for itself to serialize itself as it sees fit by defining its properties and their value objects, then any child values will get serialized according to their respective contracts.
In your converter you need to remember to write the beginning and end object tokens. If you don't you will write malformed JSON as shown.
When serializing numbers, always be sure to use the invariant culture. Utf8JsonWriter.WriteNumber() will do that correctly, but when formatting numbers manually pass NumberFormatInfo.InvariantInfo or CultureInfo.InvariantCulture to your formatting method.
It isn't clear from your question whether the Id property value should be serialized as a number or a string. If a string, use
writer.WriteString($"users__{x.ToString(NumberFormatInfo.InvariantInfo)}__id", user.Id.ToString(NumberFormatInfo.InvariantInfo));
Demo fiddle #1 here for a single converter, and #2 here for two converters.

How to deserialize an empty string to a null value for all `Nullable<T>` value types using System.Text.Json?

In .Net Core 3.1 and using System.Text.Json library, I'm facing an issue that didn't occur in Newtonsoft library.
If I send an empty string in JSON for some properties of type (type in backend) DateTime? or int?, it returns 400 status code with an error message that value can't be deserialized. However, with Newtonsoft an empty string is automatically interpreted as a null value for any Nullable<T>.
A minimal example would be:
var json = "\"\"";
Assert.AreEqual(null, Newtonsoft.Json.JsonConvert.DeserializeObject<DateTime?>(json)); // Passes
Assert.AreEqual(null, System.Text.Json.JsonSerializer.Deserialize<DateTime?>(json)); // Throws System.Text.Json.JsonException: The JSON value could not be converted to System.Nullable`1[System.DateTime].
Is there any way to make System.Text.Json behave in the same way? Demo here.

You can use the factory converter pattern to create a JsonConverterFactory that causes an empty string to be interpreted as null for all Nullable<T> type values.
The following factory does the job:
public class NullableConverterFactory : JsonConverterFactory
{
static readonly byte [] Empty = Array.Empty<byte>();
public override bool CanConvert(Type typeToConvert) => Nullable.GetUnderlyingType(typeToConvert) != null;
public override JsonConverter CreateConverter(Type type, JsonSerializerOptions options) =>
(JsonConverter)Activator.CreateInstance(
typeof(NullableConverter<>).MakeGenericType(
new Type[] { Nullable.GetUnderlyingType(type) }),
BindingFlags.Instance | BindingFlags.Public,
binder: null,
args: new object[] { options },
culture: null);
class NullableConverter<T> : JsonConverter<T?> where T : struct
{
// DO NOT CACHE the return of (JsonConverter<T>)options.GetConverter(typeof(T)) as DoubleConverter.Read() and DoubleConverter.Write()
// DO NOT WORK for nondefault values of JsonSerializerOptions.NumberHandling which was introduced in .NET 5
public NullableConverter(JsonSerializerOptions options) {}
public override T? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
if (reader.TokenType == JsonTokenType.String)
{
if (reader.ValueTextEquals(Empty))
return null;
}
return JsonSerializer.Deserialize<T>(ref reader, options);
}
public override void Write(Utf8JsonWriter writer, T? value, JsonSerializerOptions options) =>
JsonSerializer.Serialize(writer, value.Value, options);
}
}
The factory should be added to the JsonSerializerOptions.Converters collection of your framework.
Notes:
In my original version of this answer I cached the return of (JsonConverter<T>)options.GetConverter(typeof(T)) for performance as recommended by Microsoft. Unfortunately as noted in comments by zigzag Microsoft's own DoubleConverter.Read() and DoubleConverter.Write() methods do not account for non-default values of JsonSerializerOptions.NumberHandling, so I have removed this logic as of .NET 5.
Demo fiddle here.