The normal approach to serialisation is to apply attributes to your class to describe how serialisation (or deserialization) is to proceed. For example:
[DataContract]
class MyClass
{
[DataMember]
public string Name { get; set; }
}
Is there a way to perform serialisation using JSON.NET without applying attributes to your class, but instead by providing a "sidecar" object that describes what aspects of the class are to be serialised, in some fashion.
The reason I ask relates to separation of concerns. If you have an API that is meant to be agnostic about how requests get to it, then the natural extension of that is that your API data structures should not be getting embellished with serialisation attributes.
Now of course I could take the "content" of one of my API result objects and copy it into another object having a class that does have appropriate serialisation attributes, but in some cases it would seem more desirable to say "Hey, I want to serialise this object, and the object has no serialisation attributes, so here is a separate data structure to describe what to do."
The other place where this would be handy, of course, is with third-party libraries where you have no opportunity to modify the objects (again, you could make copies of the values, but I'm looking for other ways).
You can use JsonSerializerSettings to specify various serialization options. You can specify whether to serialize or not a particular property, how to serialize a particular type or convert its value, and etc.
var settings = new JsonSerializerSettings
{
Formatting = Formatting.Indented,
.....
};
settings.Converters.Add(new StringEnumConverter { CamelCaseText = true });
settings.ContractResolver = new CamelCasePropertyNamesContractResolver();
settings.Binder = new SomeSerializationBinder(new DefaultSerializationBinder());
var result = JsonConvert.SerializeObject(yourObject, settings);
Related
On our API we need to take in json, deserialize it to an interface, set a field, and ship it off. To achieve this, on both ends I'm setting the jsonConvert to use TypeNameHandling.All. The endpoint in question is supposed to be fairly locked down, but there's always a chance of someone gaining access and setting $type to a system class with a dangerous constructor or garbage collection method.
My question is would clarifying the namespace of the type before attempting to deserialize it be sufficiently safe? or would there still be a risk of having something like a sub-object with a dangerous class type in the json? If there is still a risk or an exploit I've missed, what other steps can I do to mitigate the danger?
Our company name is at the start of every namespace we use, so in the code below we just check that the type set in the json starts with our company name. The {} at the start is just so the compiler knows it doesn't need to keep the JObject in memory after the check.
{ //check the type is valid
var securityType = JsonConvert.DeserializeObject<JObject>(request.requestJson);
JToken type;
if (securityType.TryGetValue("$type", out type))
{
if (!type.ToString().ToLower().StartsWith("foo")) { //'foo' is our company name, all our namespaces start with foo
await logError($"Possible security violation, client tried to instantiate {type}", clientId: ClientId);
throw new Exception($"Request type {type} not supported, please use an IFoo");
}
}
else
{
throw new Exception("set a type...");
}
}
IFoo requestObject = JsonConvert.DeserializeObject<IFoo>(request.requestJson, new JsonSerializerSettings()
{
TypeNameHandling = TypeNameHandling.All
});
The risk with TypeNameHandling is that an attacker may trick the receiver into constructing an attack gadget - an instance of a type that when constructed, populated or disposed effects an attack on the receiving system. For an overview see
TypeNameHandling caution in Newtonsoft Json
External json vulnerable because of Json.Net TypeNameHandling auto?
If you are going to protect against such attacks by requiring all deserialized types to be in your own company's .Net namespace, be aware that, when serializing with TypeNameHandling.All, "$type" information will appear throughout the JSON token hierarchy, for all arrays and objects (including for .Net types such as List<T>). As such you must needs apply your "$type" check everywhere type information might occur. The easiest way to do this is with a custom serialization binder such as the following:
public class MySerializationBinder : DefaultSerializationBinder
{
const string MyNamespace = "foo"; //'foo' is our company name, all our namespaces start with foo
public override Type BindToType(string assemblyName, string typeName)
{
if (!typeName.StartsWith(MyNamespace, StringComparison.OrdinalIgnoreCase))
throw new JsonSerializationException($"Request type {typeName} not supported, please use an IFoo");
var type = base.BindToType(assemblyName, typeName);
return type;
}
}
Which can be used as follows:
var settings = new JsonSerializerSettings
{
SerializationBinder = new MySerializationBinder(),
TypeNameHandling = TypeNameHandling.All,
};
This has the added advantage of being more performant than your solution since pre-loading into a JObject is no longer required.
However, having done so, you may encounter the following issues:
Even if the root object is always from your company's namespace, the "$type" properties for nested values may not necessarily be in your companies namespace. Specifically, type information for harmless generic system collections such as List<T> and Dictionary<TKey, Value> as well as arrays will be included. You may need to enhance BindToType() to whitelist such types.
Serializing with TypeNameHandling.Objects or TypeNameHandling.Auto can ameliorate the need to whitelist such harmless system types, as type information for such system types is less likely to get included during serialization as compared to TypeNameHandling.All.
To further simplify the type checking as well as to reduce your attack surface overall, you might consider only allowing type information on the root object. To do that, see json.net - how to add property $type ONLY on root object. SuppressItemTypeNameContractResolver from the accepted answer can be used on the receiving side as well as the sending side, to ignore type information on non-root objects.
Alternatively, you could serialize and deserialize with TypeNameHandling.None globally and wrap your root object in a container marked with [JsonProperty(TypeNameHandling = TypeNameHandling.Auto)] like so:
public class Root<TBase>
{
[JsonProperty(TypeNameHandling = TypeNameHandling.Auto)]
public TBase Data { get; set; }
}
Since your root objects all seem to implement some interface IFoo you would serialize and deserialize a Root<IFoo> which would restrict the space of possible attack gadgets to classes implementing IFoo -- a much smaller attack surface.
Demo fiddle here.
When deserializing generics, both the outer generic and the inner generic parameter types may need to be sanitized recursively. For instance, if your namespace contains a Generic<T> then checking that the typeName begins with your company's namespace will not protect against an attack via a Generic<SomeAttackGadget>.
Even if you only allow types from your own namespace, it's hard to say that's enough to be sufficiently safe, because we don't know whether any of the classes in your own namespace might be repurposed as attack gadgets.
I have a utility that converts an xml file to a class object:
public static T CreateClassFromXml<T>(string fileName, string root) where T : class
{
fileName.ThrowNullOrEmpty("fileName");
File.Exists(fileName).ThrowFalse(string.Format("File '{0}' could not be found", fileName));
var serializer = new XmlSerializer(typeof(T), new XmlRootAttribute() { ElementName = root });
using (var reader = XmlReader.Create(fileName))
{
return (T)serializer.Deserialize(reader);
}
}
The utility reads the xml and creates a class T. Using the above code is there any way I can validate the created class other than writing a wrapper class around it? I need to ensure that data is populated for all mandatory fields.
There are no built in facilities in XmlSerializer to do this. You can do it yourself with reflection. Since XmlSerializer loads just the public properties and fields, you can iterate over all public properties and fields of the class and make sure they all hold data. You'll have to decide how to handle value types (int, DateTime, etc...) because it's not obvious when the have been loaded or not. You will also have to dive in recursively into reference types.
If you need to mark just specific properties\fields as mandatory, you can add your own attribute and decorate the class members with it. In runtime, you will only process properties which have the attribute set.
In short, unless you need a generic mechanism for many different scenarios, better do it manually for the properties you have to validate.
I have a class some thing like below
[Serializable]
public class sample
{
private int m_width;
private int m_height;
public int Width
{
get
{
return this.m_width;
}
set
{
this.m_width = value;
}
}
public int Height
{
get
{
return this.m_height;
}
set
{
this.m_height = value;
}
}
}
If I use DataContractJsonSerializer to serialize the object of this class i get the json string as below:
{"m_height":1345,"m_width":1234}
If I use Newtonsoft.Json.dll to serialize this I am getting the out put like below:
{"Width":1234,"Height":1345}
Why DataContractSerializer using backing fields for serialization if class marked as serializable ?
Is there any way I can achieve the same thing using Newtonsoft.Json.dll
We have some objects which are marked as [Serializable] so they can be serialised using traditional methods, but which we need to have cleanly serialised in JSON for use with Web API. Setting IgnoreSerializableAttribute to true will stop Newtonsoft.Json from behaving like Microsoft's serialisers and instead it will just serialise the public properties.
TLDR: Add this to WebApiConfig.cs:
((Newtonsoft.Json.Serialization.DefaultContractResolver)config.Formatters.JsonFormatter.SerializerSettings.ContractResolver).IgnoreSerializableAttribute = true;
Unless you are always communicating from WCF to WCF, your best bet is probably going to be to use the Newtonsoft serializer. Unfortunately, the MS serializer seems to follow some Microsoft-specific standards that do not match the standards that many web apps expect.
Newtonsoft's serializer seems to be more standard, and even MS uses it for WebAPI, and in the Web API http client (nuget will pull it down for you).
Here's another difference that you will find -- try serializing a DateTime type. You will find that the DataContract serialzer serializes the value in a different format that is not compatible with other JSON (you will notice some slashes in it). My understanding is that that alternate format was used by some of the AJAX WebForm controls, but it's specific to Microsoft Webforms.
Here's a little more info about the dates: http://www.hanselman.com/blog/OnTheNightmareThatIsJSONDatesPlusJSONNETAndASPNETWebAPI.aspx
Here's another thing you might look at:
The differences between DataContractJsonSerializer and Newtonsoft still stand, but as for why you are getting the weird serialization behavior -- you are mixing your serialization standards.
The [Serializable] attribute pertains to the older .Net serialization. DataContractSerialization is backward compatible, but the behavior might be different.
If you want to do this the datacontract way, mark you class with the
[DataContract] attribute instead, and mark each public member that you want to serialize with the [DataMember] attribute. (or remove all serialization attributes, and it should default to all public properties)
That should explain the difference that are seeing, but I would still recommend that you prefer the Newtonsoft serializer.
DefaultContractResolver class in Newtonsoft.Json.dll i found some code which is setting IgnoreSerializableAttribute property to true.
#if !(SILVERLIGHT || NETFX_CORE || PORTABLE || PORTABLE40)
IgnoreSerializableAttribute = true;
#endif
I am using the DotNet4.0 dll so this property set to true and it is ignoring the Serializable attribute. if i make it false it is giving the same output as DataContractSerializer
DataContract serializer requires you to explicitly mark the class with the [DataContract] attribute and then mark each desired property to be serialized with the [DataMember] attribute.
If you do that you will find that the same json string will get out as using the other class.
The issue you are encountering is due to differences of how the 2 serializers treat the default serialization (without additional information specified):
DataContract serializes all private fields unless otherwise specified
NewtonSoft serializes all public properties unless otherwise specified
On how you can make Newtonsoft serialize the private fields i have no other idea than making a wrapper class with m_Width and m_Height properties that on the setters and geters put the values in the correct target properties of the actual object
Without having spent a lot of time on this subject, I'm curious what might be the most straightforward way to allow my existing code to serialize to json in addition to xml. The existing code uses a xmlwriter to perform some sophisticated serialization for a specific purpose, so ultimately I'd like to be able to pass a jsonserializer in place of the xml one and have it produce json instead of xml.
Do any of the json libs handle something like this?
No, there's no direct drop-in replacement. The APIs for existing JSON serializers is much different. They need to have the entire model in memory and serialize it in one step. For example with JSON.NET you would use:
string json = JsonConvert.SerializeObject(product);
The correct way to achieve what you are looking for is to abstract the actual serialization of the object behind an interface:
public interface IMySerializer
{
void Serialize(MyObject instance, Stream stream);
}
Now you could have multiple implementations of this interface such as MySerializerXml, MySerializerJson, MySerializerCsv, ...
Now the code that depends on the serialization would simply work with an IMySerializer. So when you need to handle a new format all you have to do is drop-in an implementation of this interface.
There is no direct replacement for an XmlWriter. If you want to support both XML and JSON, I would suggest serialising via data-contracts, i.e. use attributes on your data objects to indicate the desired serialised format. Take a look at DataContractJsonSerializer for JSON and DataCOntractSerializer for XML.
Your model objects would look something like this:
[DataContract(Name = "Customer", Namespace = "http://www.contoso.com")]
class Person
{
[DataMember()]
public string FirstName;
[DataMember]
public string LastName;
[DataMember()]
public int ID;
}
I have an application which supports multiple types and versions of some devices. It can connect to these devices and retrieve various information.
Depending on the type of the device, I have (among other things) a class which can contain various properties. Some properties are common to all devices, some are unique to a particular device.
This data is serialized to xml.
What would be a preferred way to implement a class which would support future properties in future versions of these devices, as well as be backwards compatible with previous application versions?
I can think of several ways, but I find none of them great:
Use a collection of name-value pairs:
pros: good backward compatibility (both xml and previous versions of my app) and extensibility,
cons: no type safety, no intellisense, requires implementation of custom xml serialization (to handle different value objects)
Create derived properties class for each new device:
pros: type safety
cons: have to use XmlInclude or custom serialization to deserialize derived classes, no backward compatibility with previous xml schema (although by implementing custom serialization I could skip unknown properties?), requires casting for accessing properties in derived classes.
Another way to do it?
I am using C#, by the way.
How about something similar to a PropertyBag ?
If you're not limited to interoperability with an external schema, then you should use Runtime Serialization and the SoapFormatter. The pattern for runtime serialization permits derived classes to specify which of their properties need to be serialized and what to do with them when deserialized.
The XML Serializer requires XmlInclude because, in effect, it needs to define the schema to use.
I like name/value sets for this sort of thing.
Many of your cons can be dealt with -- consider a base class that acts as a general name/value set with no-op methods for validating incoming name/value pairs. For known sets of names (i.e. keys), you can create derived classes that implement validation methods.
For example, Printer may have a known key "PrintsColor" that can only be "true" or "false". If someone tries to load PrintsColor = "CMYK", your Printer class would throw an exception.
Depending on what you're doing, you can go a few different ways in terms of making the validation more convenient -- utility methods in the base class (e.g. checkForValidBoolean()) or a base class that accepts name/type information in its constructor for cleaner code in your derived classes, and perhaps a mostly automated XML serialization.
For intellisense -- your derived classes could have basic accessors that are implemented in terms of the key lookup. Intellisense would present those accessor names.
This approach has worked well for me -- there's sort of a short-sightedness to classic OO design, especially for large systems with plugged-in components. IMO, the clunkier type checking here is a big of a drag, but the flexibility make it worthwhile.
I believe that creating derived properties is the best choice.
You can design your new classes using xml schema. And then just generate the class code with xsd.exe.
With .net isn't hard to develop a generic class that can serialize and deserialize all types to and from xml.
public static String toXmlString<T>(T value)
{
XmlSerializer xmlSerializer = new XmlSerializer(typeof(T));
StringWriter stringWriter = new StringWriter();
try { xmlSerializer.Serialize(stringWriter, value); }
catch (Exception e)
{
throw(e);
}
finally { stringWriter.Dispose(); }
String xml = stringWriter.ToString();
stringWriter.Dispose();
return xml;
}
public static T fromXmlFile<T>(string fileName, Encoding encoding)
{
Stream stream;
try { stream = File.OpenRead(fileName); }
catch (Exception e)
{
e.Data.Add("File Name", fileName);
e.Data.Add("Type", typeof(T).ToString());
throw(e);
}
BufferedStream bufferedStream = new BufferedStream(stream);
XmlSerializer xmlSerializer = new XmlSerializer(typeof(T));
TextReader textReader;
if (encoding == null)
textReader = new StreamReader(bufferedStream);
else
textReader = new StreamReader(bufferedStream, encoding);
T value;
try { value = (T)xmlSerializer.Deserialize(textReader); }
catch (Exception e)
{
e.Data.Add("File Name", fileName);
e.Data.Add("Type", typeof(T).ToString());
throw(e);
}
finally
{
textReader.Dispose();
bufferedStream.Dispose();
}
return value;
}
Programatically speaking, this sounds like it might be a job for the Decorator Pattern. Essentially, you have a super class which defines a common interface for all these types of devices. Then you have decorator classes which have other properties which a device might have. And, when creating these devices, you can dynamically add these decorations to define new properties for the device. Graphically:
You can look at the Wikipedia page for a more detailed description. After that, it would just be a matter of doign some serialization to tell the program which decorators to load.
The general idea of what you're trying to accomplish here is precisely what the EAV pattern solves. EAV is a pattern most commonly used in database development but the concept is equally valid for applications.