Wrote a Custom JsonConverter to handle different Json formats that are returned by different versions of the same api. One app makes a request to several other apps, and we dont know which format will be returned so the JsonConverter handles this and seems to work well. I need to add unit tests to the project, except I have not found helpful resources to help Mock out some of the Newtonsoft.Json objects, mainly JsonReader.
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JObject jsonValue = JObject.Load(reader);
if(jsonValue == null)
{
return null;
}
var responseData = ReadJsonObject(jsonValue);
return responseData;
}
[TestMethod]
public void ReadJsonReturnNullForNullJson()
{
var converter = new DataConverter();
_mockJsonReader.Setup(x => x.Value).Returns(null);
var responseData = converter.ReadJson(_mockJsonReader.Object, typeof(ProbeResponseData), null, _mockJsonSerializer.Object);
Assert.IsNull(responseData);
}
Some code has been taken out of the ReadJson method. I am trying Setup the JsonReader to return the value of the actual json, in this case a null value but in other unit tests I would want an actual Json(JObject). When running the unit test I receive a "Newtonsoft.JsonReaderException: Error reading JObject from JsonReader. Path ''."
The use of DeserializeObject<T> will call your override of ReadJson under the hood.
[TestMethod]
public void ReadJsonVerifyTypeReturned()
{
var testJson = CreateJsonString();
var result = JsonConvert.DeserializeObject<ProbeResponseData>(testJson);
var resultCheck = result as ProbeResponseData;
Assert.IsNotNull(resultCheck);
}
Whilst using JsonConvert or JsonSerializer directly will allow you to test it, you probably should make your converter tests a little more direct. For instance, you can't guarantee that JSON.NET will do what you expect when you call the deserializer, whereas what you actually want to test is your custom converter - what JSON.NET does with that is out of your control.
Consider this example:
public readonly struct UserId
{
public static readonly UserId Empty = new UserId();
public UserId(int value)
{
Value = value;
HasValue = true;
}
public int Value { get; }
public bool HasValue { get; }
}
I've got this struct, which is backed by an int. I want to deserialize a specific JSON number value as an int -> UserId. So, I create a custom converter:
public class UserIdConverter
{
public override bool CanConvert(Type objectType) => objectType == typeof(UserId);
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
int? id = serializer.Deserialize<int?>(reader);
if (!id.HasValue)
{
return UserId.Empty;
}
return new UserId(id.Value);
}
}
I've skipped over the implementation of WriteJson in this instance, but the logic is the same.
I would write my test as follows:
[Fact]
public void UserIdJsonConverter_CanConvertFromJsonNumber()
{
// Arrange
var serialiser = new JsonSerializer();
var reader = CreateJsonReader("10");
var converter = new UserIdJsonConverter();
// Act
var result = converter.ReadJson(reader, typeof(UserId), null, serialiser);
// Assert
Assert.NotNull(result);
Assert.IsType<UserId>(result);
var id = (UserId)result;
Assert.True(id.HasValue);
Assert.Equal(10, id.Value);
}
private JsonTextReader CreateJsonReader(string json)
=> new JsonTextReader(new StringReader(json));
In doing so, I can create a test purely around my ReadJson method, and confirm it does what I expect. Going further, I could potentially mock elements, such as the JsonReader and JsonSerializer to result in different preconditions so I can test a wide array of scenarios.
The issue with relying on JsonConvert or JsonSerializer to run the full deserialization process, is that you're introducing other logic which is largely outside of your control. I.e., what if through deserialization, JSON.NET actually makes a different decision and your custom converter is never used - your test isn't responsible for testing JSON.NET itself, but what your custom converter actually does.
Without knowing the inner workings of your custom converter, could you just create a test double instead of a mock?
public class JsonReaderThatReturnsNull : JsonReader
{
public override bool Read()
{
return true;
}
public override object Value => null;
}
To get to test the converter, it needs to be in the list of converters. If you used JsonConverterAttribute, it will work automatigically, but if you didn't you can do it like this:
var serializerSettings = new JsonSerializerSettings();
serializerSettings.Converters.Add(new MyJsonConverter());
var serializer = JsonSerializer.Create(serializerSettings);
var jObject = JObject.Parse(myString);
var result = jObject.ToObject<MyObject>(serializer);
Try to use this to check converted result.
DeserializedType result = JsonConvert.DeserializeObject<DeserializedType>(json, new Converter(parms));
From Documentation
Related
The details of the problem might be a little long, so I'll describe it in short at the beginning: How to force Json.Net to use its default object serializer(or ignore a specific custom converter in other words), but still keep the settings in a JsonSerializer, when deserializing an object?
Apologize for my poor English, the description may be kind of ambiguous and confusing. I'll explain it with my detailed scenario.
When dealing with HTTP responses, we sometimes encounter a scenario that an object is the only child of its parent, making the parent object a meaningless object wrapper to some extent. In some poor designs, there could be multiple levels of such wrappers. If we want such JSON deserialized properly without customizing, we have to follow the structure to define those wrapper classes, which is definitely pointless and annoying, thus I came up with the idea to create a general-purpose ObjectWrapperConverter. Here's the code:
public class ObjectWrapperConverter<T> : ObjectWrapperConverterBase<T> {
public ObjectWrapperConverter(string propertyName) : this(propertyName, Array.Empty<JsonConverter>()) { }
public ObjectWrapperConverter(string propertyName, params JsonConverter[] converters) {
PropertyName = propertyName;
Converters = converters;
}
public override string PropertyName { get; }
public override JsonConverter[] Converters { get; }
}
public abstract class ObjectWrapperConverterBase<T> : JsonConverter<T> {
public abstract string PropertyName { get; }
public abstract JsonConverter[] Converters { get; }
public sealed override void WriteJson(JsonWriter writer, T value, JsonSerializer serializer) {
writer.WriteStartObject();
writer.WritePropertyName(PropertyName);
serializer.Converters.AddRange(Converters);
writer.WriteValue(value, serializer);
serializer.Converters.RemoveRange(Converters);
writer.WriteEndObject();
}
public sealed override T ReadJson(JsonReader reader, Type objectType, T existingValue, bool hasExistingValue, JsonSerializer serializer) {
var token = JToken.Load(reader);
if (token.Type != JTokenType.Object)
throw new JTokenTypeException(token, JTokenType.Object);
var obj = token as JObject;
var prop = obj!.Property(PropertyName);
if (prop is null)
throw new JTokenException(token, $"Property \"{PropertyName}\" not found");
serializer.Converters.AddRange(Converters);
var result = prop.Value.ToObject<T>(serializer);//BUG: recurse when applying JsonConverterAttribute to a class
serializer.Converters.RemoveRange(Converters);
return result;
}
}
It works fine when I put JsonConverterAttribute on properties and fields. But when annotating class, problem occurs: the deserialization process fall into a recursive loop.
I debugged into Json.Net framework, and realized that when specifying a custom converter for a class, Json.Net will always use this converter to handle the serialization of this class unless higher-priority attribute (like JsonConverterAttribute placed on properties) is annotated. Thus, in my converter, the line where I put a comment will finally lead to a recurse.
If you've understood the purpose of this converter, it's easy to find out that this converter is just a middleware: add or remove the wrapper object, and continue the original serialization process.
So, how can I continue the "original" serialization process instead of falling into the converter itself again?
I made a deeper exploration into the Newtonsoft.Json framework and found out how it serialize and deserialize objects without converters.
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue
Because these classes and methods are internal, I have to use Reflection to invoke them. Thus I encapsulate this into two extension methods:
public static class NewtonsoftExtensions{
private static Type JsonSerializerInternalReader { get; } = typeof(JsonSerializer).Assembly.GetType("Newtonsoft.Json.Serialization.JsonSerializerInternalReader");
private static Type JsonSerializerInternalWriter { get; } = typeof(JsonSerializer).Assembly.GetType("Newtonsoft.Json.Serialization.JsonSerializerInternalWriter");
private static MethodInfo CreateValueInternal { get; } = JsonSerializerInternalReader.GetMethod("CreateValueInternal", BindingFlags.NonPublic | BindingFlags.Instance);
private static MethodInfo SerializeValue { get; } = JsonSerializerInternalWriter.GetMethod("SerializeValue", BindingFlags.NonPublic | BindingFlags.Instance);
public object DeserializeWithoutContractConverter(this JsonSerializer serializer, JsonReader reader, Type objectType) {
var contract = serializer.ContractResolver.ResolveContract(objectType);
var converter = contract.Converter;
contract.Converter = null;
object internalReader = Activator.CreateInstance(JsonSerializerInternalReader, serializer);
object result = CreateValueInternal.Invoke(internalReader, reader, objectType, contract, null, null, null, null);
contract.Converter = converter; //DefaultContractResolver caches the contract of each type, thus we need to restore the original converter for future use
return result;
}
public void SerializeWithoutContractConverter(this JsonSerializer serializer, JsonWriter writer, object value) {
var contract = serializer.ContractResolver.ResolveContract(value.GetType());
var converter = contract.Converter;
contract.Converter = null;
object internalWriter = Activator.CreateInstance(JsonSerializerInternalWriter, serializer);
SerializeValue.Invoke(internalWriter, writer, value, contract, null, null, null);
contract.Converter = converter;
}
}
Using reflection to call internal methods is risky and should not be recommended, but compared with other answers in JSON.Net throws StackOverflowException when using [JsonConvert()], such approach would make full use of serializer settings. If the converter is general-purpose, like the ObjectWrapperConverter I'm trying to implement, this will cause least the unexpected results, as Newtonsoft.Json has tons of settings for users to customize the behaviors.
Suppose I have these classes:
public class Bar
{
public Foo MyFoo { get; set; }
}
public class Foo
{
public string[] Stuff { get; set; }
}
And I have this JSON structure:
{
"MyFoo":
{
"Stuff":"those,are,my,stuff"
}
}
And I have a code path where a JObject is being converted to Bar using code like below:
myJObject.ToObject(typeof(Bar))
Now what I need to do is to supply the ToObject with a custom serializer to convert the string property Stuff into an array of string (using string.Split(...).ToArray())
I was asked not to add attributes on the client class 'Bar' so after looking around it seemed like a ContractResolver is in order but the issue is that the resolver only lets me handle direct properties of the root Type, that is Bar in my example, and I can't register a JsonConverter on a nested property.
So my question to you guys is, is this even achievable using Json.net?
Note that I need to do this not only for the Bar class but to an unlimited amount of classes with unknown structure so I can't hard-code a solution that will work for one type of class.
Based on your description, I don't see why you would need a ContractResolver. You know that the properties you want to deserialize specially will always be of type string[], so just make a converter that handles that type. Maybe something like this:
public class CsvStringConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(string[]);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JToken token = JToken.Load(reader);
if (token.Type == JTokenType.Null)
return null;
if (token.Type == JTokenType.String)
return ((string)token).Split(',');
if (token.Type == JTokenType.Array)
return token.ToObject<string[]>(serializer);
throw new JsonException("Unexpected token type: " + token.Type);
}
public override bool CanWrite => false;
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Then, to use it with your JObject, create a new JsonSerializer instance, add the converter to it, and pass the serializer to the ToObject() method like this:
var serializer = new JsonSerializer();
serializer.Converters.Add(new CsvStringConverter());
var bar = myJObject.ToObject<Bar>(serializer);
Working demo here: https://dotnetfiddle.net/qmeBoh
I mock an interface from which the object is serialized by the tested code by the Newtonsoft JsonConvert.SerializeObject.
The serializer throws an exception with the following error:
Newtonsoft.Json.JsonSerializationException : Self referencing loop detected for property 'Object' with type 'Castle.Proxies.IActionProxy'. Path 'Mock'.
Newtonsoft tries to serialized the proxy object IActionProxy, that has a property Mock that loops on that serialized object.
Curiously changing serializer options
ReferenceLoopHandling = ReferenceLoopHandling.Serialize ( ot Ignore)
PreserveReferencesHandling = PreserveReferencesHandling.All
..does not solve the problem, serialization become infinite
Thank you for help about this issue, I would be happy to have a way of using Moq in that case
UPDATE: here is a sample code to produce the exception:
Mock<IAction> _actionMock = new Mock<IAction>().SetupAllProperties();
Newtonsoft.Json.JsonConvert.SerializeObject( _actionMock.Object ); // JsonSerializationException (this line is in a method which i'm not responsible of )
// IAction is any interface with some properties
We have to consider the serialization (SerializeObject) is called by the tested code in a library I don't have access to.
This is a little rough around the edges, but it does the job:
public class JsonMockConverter : JsonConverter {
static readonly Dictionary<object, Func<object>> mockSerializers = new Dictionary<object, Func<object>>();
static readonly HashSet<Type> mockTypes = new HashSet<Type>();
public static void RegisterMock<T>(Mock<T> mock, Func<object> serializer) where T : class {
mockSerializers[mock.Object] = serializer;
mockTypes.Add(mock.Object.GetType());
}
public override bool CanConvert(Type objectType) => mockTypes.Contains(objectType);
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) =>
throw new NotImplementedException();
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
if (!mockSerializers.TryGetValue(value, out var mockSerializer)) {
throw new InvalidOperationException("Attempt to serialize unregistered mock.");
}
serializer.Serialize(writer, mockSerializer());
}
}
A little extension method for ease of use:
internal static class MockExtensions {
public static Mock<T> RegisterForJsonSerialization<T>(this Mock<T> mock) where T : class {
JsonMockConverter.RegisterMock(
mock,
() => typeof(T).GetProperties().ToDictionary(p => p.Name, p => p.GetValue(mock.Object))
);
return mock;
}
}
Set up as follows:
JsonConvert.DefaultSettings = () => new JsonSerializerSettings {
Converters = new[] { new JsonMockConverter() }
};
And now the following code works:
public interface IAction {
int IntProperty { get; set; }
}
var actionMock = new Mock<IAction>()
.SetupAllProperties()
.RegisterForJsonSerialization();
var action = actionMock.Object;
action.IntProperty = 42;
Console.WriteLine(JsonConvert.SerializeObject(action));
Making it so that you don't have to register your mocks for serialization is much harder -- there is no robust way of figuring out that an object is a mock, and if it is, what type it's supposed to be mocking, and if it is, how we're supposed to serialize that using the mock's data. This could only be done with some really nasty and brittle reflection over Moq's internals, but let's not go there. It could possibly be added to Moq itself as a feature, though.
This can be further extended with custom ways of serializing -- here I've assumed we're OK with just serializing the public properties of the mocked type. This is a little naive -- it will not work correctly if you are inheriting interfaces, for example, because Type.GetProperties only gets the properties declared on the interface itself. Fixing that, if desired, is left as an exercise to the reader.
Extending this for deserialization is possible in principle, but a bit trickier. It would be very unusual to need that for mocking purposes, as opposed to a concrete instance.
If you're not worrying about JsonConvert.SerializeObject() result, you can just simple add the following line into your code:
_actionMock.As<ISerializable>();
I have created an implementation of JsonConverter
CanConvert always returns true.
In ReadJson I want to sometimes just use the "default" behavior, as if CanConvert had returned false and my ReadJson was never called.
Various other posts have suggested I do some variation of the following:
existingValue = existingValue ?? serializer
.ContractResolver
.ResolveContract(objectType)
.DefaultCreator();
serializer.Populate(reader, existingValue);
However, this throws NullReferenceException on .DefaultCreator().
existingValue is always null
The ContractResolver that is returned from the serializer is my own. It extends json.net's built-in CamelCasePropertyNamesContractResolver and simply overrides the methods CreateConstructorParameters and CreatePropertyFromConstructorParameter
How do I tell json.net - "just kidding, I don't know how to create this thing, do whatever you would have done to create it had I told you that I couldn't create it"
Note that I've simplified the problem for discussion. I am anticipating someone will answer with "just have CanCreate return false" In fact, in a number of scenarios I can and should create the object.
From your question and comments, it sounds like you have some situations where you want a converter to read but not write, and others where you want it to write but not read. You've solved the problem by splitting the functionality into two converters and then having each converter's CanConvert method return true or false at the appropriate times. This is certainly a viable approach and seems to be working for you, which is great. However, I wanted to offer an alternative solution.
In addition to the CanConvert method, the base JsonConverter offers two virtual boolean properties which you can override: CanRead and CanWrite. (Both return true by default.) These properties directly control whether ReadJson and WriteJson are called by the serializer for a particular converter. So, for example, if CanRead returns false, then ReadJson will not be called and the default read behavior will be used instead, even though CanConvert returned true. This allows you to set up an asymmetric converter quite neatly. For example, you might have a situation where you want to deserialize a crazy JSON format into a more sane object structure, but when you serialize it again, you don't want to go back to the crazy JSON format-- you just want the default serialization. In that case you would override CanWrite in your converter to always return false. Then you could just leave the implementation of WriteJson blank or have it throw a NotImplementedException; it will never be called.
Your case sounds a little more complicated than that, but you still should be able to manipulate the CanRead and CanWrite properties to achieve your desired results. Below is a contrived example which shows how we can switch the ReadJson and WriteJson methods on and off depending on a situational variable.
public class Program
{
public static void Main(string[] args)
{
string json = #"{""keys"":[""foo"",""fizz""],""values"":[""bar"",""bang""]}";
CustomConverter converter = new CustomConverter();
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.Converters.Add(converter);
// Here we are reading a JSON object containing two arrays into a dictionary
// (custom read) and then writing out the dictionary JSON (standard write)
Console.WriteLine("--- Situation 1 (custom read, standard write) ---");
converter.Behavior = ConverterBehavior.CustomReadStandardWrite;
json = DeserializeThenSerialize(json, settings);
// Here we are reading a simple JSON object into a dictionary (standard read)
// and then writing out a new JSON object containing arrays (custom write)
Console.WriteLine("--- Situation 2 (standard read, custom write) ---");
converter.Behavior = ConverterBehavior.StandardReadCustomWrite;
json = DeserializeThenSerialize(json, settings);
}
private static string DeserializeThenSerialize(string json, JsonSerializerSettings settings)
{
Console.WriteLine("Deserializing...");
Console.WriteLine(json);
var dict = JsonConvert.DeserializeObject<Dictionary<string, string>>(json, settings);
foreach (var kvp in dict)
{
Console.WriteLine(kvp.Key + ": " + kvp.Value);
}
Console.WriteLine("Serializing...");
json = JsonConvert.SerializeObject(dict, settings);
Console.WriteLine(json);
Console.WriteLine();
return json;
}
}
enum ConverterBehavior { CustomReadStandardWrite, StandardReadCustomWrite }
class CustomConverter : JsonConverter
{
public ConverterBehavior Behavior { get; set; }
public override bool CanConvert(Type objectType)
{
return typeof(IDictionary<string, string>).IsAssignableFrom(objectType);
}
public override bool CanRead
{
get { return Behavior == ConverterBehavior.CustomReadStandardWrite; }
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
Console.WriteLine("ReadJson was called");
// Converts a JSON object containing a keys array and a values array
// into a Dictionary<string, string>
JObject jo = JObject.Load(reader);
return jo["keys"].Zip(jo["values"], (k, v) => new JProperty((string)k, v))
.ToDictionary(jp => jp.Name, jp => (string)jp.Value);
}
public override bool CanWrite
{
get { return Behavior == ConverterBehavior.StandardReadCustomWrite; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
Console.WriteLine("WriteJson was called");
// Converts a dictionary to a JSON object containing
// a keys array and a values array from the dictionary
var dict = (Dictionary<string, string>)value;
JObject jo = new JObject(new JProperty("keys", new JArray(dict.Keys)),
new JProperty("values", new JArray(dict.Values)));
jo.WriteTo(writer);
}
}
Output:
--- Situation 1 (custom read, standard write) ---
Deserializing...
{"keys":["foo","fizz"],"values":["bar","bang"]}
ReadJson was called
foo: bar
fizz: bang
Serializing...
{"foo":"bar","fizz":"bang"}
--- Situation 2 (standard read, custom write) ---
Deserializing...
{"foo":"bar","fizz":"bang"}
foo: bar
fizz: bang
Serializing...
WriteJson was called
{"keys":["foo","fizz"],"values":["bar","bang"]}
Fiddle: https://dotnetfiddle.net/BdtSoN
Using Json.Net, I have properties in my objects which need special care in order to serialize / deserialize them. Making a descendant of JsonConverter, I managed to accomplish this successfully. This is the common way of doing this:
public class SomeConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
...
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
...
}
public override bool CanConvert(Type objectType)
{
...
}
}
class SomeClass
{
[JsonProperty, JsonConverter(typeof(SomeConverter))]
public SomeType SomeProperty;
}
//Later on, in code somewhere
SomeClass SomeObject = new SomeClass();
string json = JsonConvert.SerializeObject(SomeObject, new SomeConverter());
My problem with this code is that I need to introduce my custom converter in every serialization / deserialization. In my project there are many cases that I cannot do that. For instance, I'm using other external projects which make use of Json.Net as well and they will be working on my SomeClass instances. But since I don't want to or can't make change in their code, I have no way to introduce my converter.
Is there any way I can register my converter, using some static member perhaps, in Json.Net so no matter where serialization / deserialization happens, my converter is always present?
Yes, this is possible using Json.Net 5.0.5 or later. See JsonConvert.DefaultSettings.
JsonConvert.DefaultSettings = () => new JsonSerializerSettings
{
Converters = new List<JsonConverter> { new SomeConverter() }
};
// Later on...
string json = JsonConvert.SerializeObject(someObject); // this will use SomeConverter
If you're using Web API, you can set up a converter globally like this instead:
var config = GlobalConfiguration.Configuration;
var jsonSettings = config.Formatters.JsonFormatter.SerializerSettings;
jsonSettings.Converters.Add(new SomeConverter());
Another approach (which wins in priority over the one #Brian mentions above) is to implement a custom contract resolver
JsonFormatter.SerializerSettings.ContractResolver = new CustomContractResolver();
And the implementation is rather straightforward
public class CustomContractResolver : DefaultContractResolver
{
private static readonly JsonConverter _converter = new MyCustomConverter();
private static Type _type = typeof (MyCustomType);
protected override JsonConverter ResolveContractConverter(Type objectType)
{
if (objectType == null || !_type.IsAssignableFrom(objectType)) // alternatively _type == objectType
{
return base.ResolveContractConverter(objectType);
}
return _converter;
}
}
Both methods are valid, this one is just a bigger hammer
This approach from the question ASP.NET Web API Custom JsonConverter is never called works with Web API:
// Add a custom converter for Entities.
foreach (var formatter in GlobalConfiguration.Configuration.Formatters)
{
var jsonFormatter = formatter as JsonMediaTypeFormatter;
if (jsonFormatter == null)
continue;
jsonFormatter.SerializerSettings.Converters.Add(new MyConverter());
}
Just put it somewhere into Global.asax.
The other answers didn't work for me. DefaultSettings has no effect on Web API actions, and the JsonFormatter configuration property does not seem to exist in the .NET framework version I use.