We Keep Coding

Serialization/Deserialization without attributes in C#

I have a host of classes including the normal designs of object hierarchies and interfaces, base classes, etc. from a project where I cannot modify any code. I have another payload class in my project which uses composition to encapsulate information from the other classes and contain properties in the payload class whose types are the classes from the other project.
Now I have a need to be able to create an instance of the payload class containing instances of those other classes and serialize it to Base64 string for transmission.
Issue is since I cannot touch the code for the other classes, I cannot add serialization attributes (for .NET binary formatter/protobuf-net). I was also trying to look at using protobuf-net without attributes, but since the object hierarchy is too deep, it seemed to be too complicated to create.
Can somebody tell me a better choice to go ahead with the serialization/deserialization part without modifying the existing code.
Sample code to illustrate the requirement is shown below:
void Main()
{
var b = new B();
b.SetStatus("This is B");
var c = new C { Name = "C", Value = 100};
var payload = new Payload(b, c);
var serializedData = SerializeToString<Payload>(payload);
serializedData.Dump();
}
private static TData DeserializeFromString<TData>(string settings)
{
byte[] b = Convert.FromBase64String(settings);
using (var stream = new MemoryStream(b))
{
var formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
return (TData)formatter.Deserialize(stream);
}
}
private static string SerializeToString<TData>(TData settings)
{
using (var stream = new MemoryStream())
{
var formatter = new BinaryFormatter();
formatter.Serialize(stream, settings);
stream.Flush();
stream.Position = 0;
return Convert.ToBase64String(stream.ToArray());
}
}
public class A
{
public string Id { get; set; }
public string StatusMsg {get;protected set;}
public virtual void SetStatus(string msg)
{
StatusMsg = msg;
}
}
public class B : A
{
public B()
{
Id = new Guid().ToString();
}
public override void SetStatus(string msg)
{
base.SetStatus(msg);
}
}
public class C
{
public string Name
{
get;
set;
}
public Int32 Value
{
get;
set;
}
}
public class Payload
{
public B PropertyB { get; set; }
public C PropertyC { get; set; }
public Payload(B b, C c)
{
this.PropertyB = b;
this.PropertyC = c;
}
}

Without adding configuration attributes, you have a few options:
use a serialize that won't care: XmlSerializer or Json.NET might work if you're lucky
accept the runtime config work of something like protobuf-net or one of the others; it probably isn't as much work as you expect (heck, drop me an email with real code and I might be able to do it)
wrote the serialization entirely manually
write a DTO layer - a basic model with attributes etc that works well with your chosen serializer - and either write code to map between the two models, or use a tool like auto-mapper
Personally, when serialization configuration gets tricky, my default option is "write a DTO model".

I don't understand why protobuf would not work but you can do the serialization manually if you don't mind.
In that case, create a BinaryWriter and then write everything you need to be able to deserialize it back again using a BinaryReader. Doing it this way you will get a very compact representation tailored for you specific needs which is also very fast. But it's a bit more work also. General purpose serialization can be verbose and slow but is almost always preferred in any case.
But I still don't get why using any existing attribute-less serialization method wouldn't work. You should probably start by really understand why it doesn't work with for instance protobuf or JSON.NET. You mentioned "too deep hierarchy". How deep is that?

Protobuf with many unknown types

What is the best approach when I do not know at runtime which types are de/serialized with protobuf?
Currently I am playing with the idea to extend the RuntimeTypeModel in the type initializers of the types which are candidates for serialization which seems to work pretty well for serialiation. But when deserializing in a different process I would need to load the same type model from somewhere which was used to serialize the types. Is it possible to serialize the RuntimeTypeModel to disk to reuse it later when the serialized data is read again from disk? Ideally I would put the model into the serialized stream as well to have a full self describing object model. Or would I need to record the steps and put this data in front of my serialized stream?
One could create a header which contains the offset to the real data and the runtime type model and the length which would be pretty nice. Or is there a better approach how to deal with a plug in architecture where at serialization time I have all types registered but during deseralization I might stil need to load some types from their respective assemblies because the code was not yet touched?
using ProtoBuf;
using ProtoBuf.Meta;
using System.Collections.Generic;
using System.IO;
namespace protobuf
{
[ProtoContract]
public interface IAbstraction
{
[ProtoMember(1)]
string Name { get; set; }
}
[ProtoContract]
public class Base : IAbstraction
{
static Base()
{
ProtobufTypeModels.MainModel.Add(typeof(IAbstraction), true).AddSubType(101, typeof(Base));
}
[ProtoMember(1)]
public string Name { get; set; }
[ProtoMember(2, AsReference =true)]
public List<IAbstraction> Instances = new List<IAbstraction>();
}
[ProtoContract]
public class Next : Base
{
static Next()
{
ProtobufTypeModels.MainModel.Add(typeof(IAbstraction), true).AddSubType(100, typeof(Next));
}
[ProtoMember(1)]
public string NextName { get; set; }
}
public static class ProtobufTypeModels
{
public static readonly RuntimeTypeModel MainModel = TypeModel.Create();
}
class Program
{
static void Main(string[] args)
{
Base b = new Base { Name = "Alois" };
b.Instances.Add(new Next { Name = "Base", NextName = "Christian" });
b.Instances.Add(new Base { Name = "SecondBase", Instances = b.Instances });
var mem = new MemoryStream();
ProtobufTypeModels.MainModel.Serialize(mem, b);
mem.Position = 0;
var deser = (Base) ProtobufTypeModels.MainModel.Deserialize(mem, null, typeof(Base));
}
}
}

Random thought: you could use .Compile(serializerName,dllPath) after you've finished the initialization and write the baked serializer to disk; then you can reference it, use new SerializerName() to create the instance, and use the .Serialize etc methods from there. The dll will never change. This also means it never has to process any metadata ever again; no reflection, no IL emit, etc.
Other than that: we could possibly do something more gentle in terms of storing the configuration, but: protobuf-net doesn't currently add anything directly to support it, and it would probably be more relevant for you to have your own bespoke configuration data that you simply consume at startup.

Determining object "type" during json deserialization

While trying to de-serialize a complex JSON object (JIRA issue) into an object containing a dictionary of type string-Field I've hit a bit of a bump.
While I can de-serialize various pre-determined object types (standard), I'm having a bit of a harder time with the custom fields, which could be of various types (they all begin with customfield_ followed by a set of numbers).
The custom fields can be floats, strings, booleans, objects and arrays of objects. The latter of these is causing me issues since I can't seem to determine what the object is before I de-serialize it.
I've searched for a way to perhaps "peek" at the data in the object before de-serializing as one of the fields contains information specific to it's type. This is all so I can determine the type of the object and tell Json.Net what to de-serialize it as.
I've considered parsing the JSON string before serialization to get the information, or maybe just when hitting this particular case, but maybe there is a better way?
Thanks in advance for any advice on this.

You can deserialize to an object with Json.Net. Here's a quick and dirty example:
using System;
using Newtonsoft.Json;
namespace Sandbox
{
class Program
{
private static void Main(string[] args)
{
var nestDto = new Dto
{
customfield_1 = 20,
customfield_2 = "Test2"
};
var dto = new Dto
{
customfield_1 = 10,
customfield_3 = new[] { nestDto },
customfield_2 = "Test"
};
var jsonString = JsonConvert.SerializeObject(dto);
Console.WriteLine(jsonString);
var fromJsonString = JsonConvert.DeserializeObject<Dto>(jsonString);
Console.WriteLine(fromJsonString.customfield_3[0].customfield_2); //Outputs Test2
Console.ReadKey();
}
}
class Dto
{
public int customfield_1 { get; set; }
public string customfield_2 { get; set; }
public Dto[] customfield_3 { get; set; }
}
}

Instead of peaking, you can deserialize as the same type as JSON.net uses for ExtensionData explicitly. For example:
if (reader.TokenType == JsonToken.StartArray)
{
var values = serializer.Deserialize<List<Dictionary<string, JToken>>>(reader);
objectContainer = ClassifyAndReturn(values);
}
private ObjectType ClassifyAndReturn(List<Dictionary<string, JToken>> values)
{
if (values.First().ContainsKey("self"))
{
string self = values.First()["self"].Value<string>();
if (self.Contains("customFieldOption"))
//... Then go into a series of if else cases to determine the object.
The representation of the objects are given as a Dictionary of string to JToken, which can then easily be checked and assigned manually or in some cases automatically deserialized (in the case one of the fields is another object).
Here is what an object constructor could look like:
internal myobject(Dictionary<string, JToken> source)
{
Self = source["self"].Value<string>();
Id = source["id"].Value<string>();
Value = source["value"].Value<string>();
}

Serializing a list to JSON

I have an object model that looks like this:
public MyObjectInJson
{
public long ObjectID {get;set;}
public string ObjectInJson {get;set;}
}
The property ObjectInJson is an already serialized version an object that contains nested lists. For the moment, I'm serializing the list of MyObjectInJson manually like this:
StringBuilder TheListBuilder = new StringBuilder();
TheListBuilder.Append("[");
int TheCounter = 0;
foreach (MyObjectInJson TheObject in TheList)
{
TheCounter++;
TheListBuilder.Append(TheObject.ObjectInJson);
if (TheCounter != TheList.Count())
{
TheListBuilder.Append(",");
}
}
TheListBuilder.Append("]");
return TheListBuilder.ToString();
I wonder if I can replace this sort of dangerous code with JavascriptSerializer and get the same results.
How would I do this?

If using .Net Core 3.0 or later;
Default to using the built in System.Text.Json parser implementation.
e.g.
using System.Text.Json;
var json = JsonSerializer.Serialize(aList);
alternatively, other, less mainstream options are available like Utf8Json parser and Jil: These may offer superior performance, if you really need it but, you will need to install their respective packages.
If stuck using .Net Core 2.2 or earlier;
Default to using Newtonsoft JSON.Net as your first choice JSON Parser.
e.g.
using Newtonsoft.Json;
var json = JsonConvert.SerializeObject(aList);
you may need to install the package first.
PM> Install-Package Newtonsoft.Json
For more details see and upvote the answer that is the source of this information.
For reference only, this was the original answer, many years ago;
// you need to reference System.Web.Extensions
using System.Web.Script.Serialization;
var jsonSerialiser = new JavaScriptSerializer();
var json = jsonSerialiser.Serialize(aList);

You can also use Json.NET. Just download it at http://james.newtonking.com/pages/json-net.aspx, extract the compressed file and add it as a reference.
Then just serialize the list (or whatever object you want) with the following:
using Newtonsoft.Json;
string json = JsonConvert.SerializeObject(listTop10);
Update: you can also add it to your project via the NuGet Package Manager (Tools --> NuGet Package Manager --> Package Manager Console):
PM> Install-Package Newtonsoft.Json
Documentation: Serializing Collections

There are two common ways of doing that with built-in JSON serializers:
JavaScriptSerializer
var serializer = new JavaScriptSerializer();
return serializer.Serialize(TheList);
DataContractJsonSerializer
var serializer = new DataContractJsonSerializer(TheList.GetType());
using (var stream = new MemoryStream())
{
serializer.WriteObject(stream, TheList);
using (var sr = new StreamReader(stream))
{
return sr.ReadToEnd();
}
}
Note, that this option requires definition of a data contract for your class:
[DataContract]
public class MyObjectInJson
{
[DataMember]
public long ObjectID {get;set;}
[DataMember]
public string ObjectInJson {get;set;}
}

public static string JSONSerialize<T>(T obj)
{
string retVal = String.Empty;
using (MemoryStream ms = new MemoryStream())
{
DataContractJsonSerializer serializer = new DataContractJsonSerializer(obj.GetType());
serializer.WriteObject(ms, obj);
var byteArray = ms.ToArray();
retVal = Encoding.UTF8.GetString(byteArray, 0, byteArray.Length);
}
return retVal;
}

.NET already supports basic Json serialization through the System.Runtime.Serialization.Json namespace and the DataContractJsonSerializer class since version 3.5. As the name implies, DataContractJsonSerializer takes into account any data annotations you add to your objects to create the final Json output.
That can be handy if you already have annotated data classes that you want to serialize Json to a stream, as described in How To: Serialize and Deserialize JSON Data. There are limitations but it's good enough and fast enough if you have basic needs and don't want to add Yet Another Library to your project.
The following code serializea a list to the console output stream. As you see it is a bit more verbose than Json.NET and not type-safe (ie no generics)
var list = new List<string> {"a", "b", "c", "d"};
using(var output = Console.OpenStandardOutput())
{
var writer = new DataContractJsonSerializer(typeof (List<string>));
writer.WriteObject(output,list);
}
On the other hand, Json.NET provides much better control over how you generate Json. This will come in VERY handy when you have to map javascript-friendly names names to .NET classes, format dates to json etc.
Another option is ServiceStack.Text, part of the ServicStack ... stack, which provides a set of very fast serializers for Json, JSV and CSV.

building on an answer from another posting.. I've come up with a more generic way to build out a list, utilizing dynamic retrieval with Json.NET version 12.x
using Newtonsoft.Json;
static class JsonObj
{
/// <summary>
/// Deserializes a json file into an object list
/// Author: Joseph Poirier 2/26/2019
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="fileName"></param>
/// <returns></returns>
public static List<T> DeSerializeObject<T>(string fileName)
{
List<T> objectOut = new List<T>();
if (string.IsNullOrEmpty(fileName)) { return objectOut; }
try
{
// reading in full file as text
string ss = File.ReadAllText(fileName);
// went with <dynamic> over <T> or <List<T>> to avoid error..
// unexpected character at line 1 column 2
var output = JsonConvert.DeserializeObject<dynamic>(ss);
foreach (var Record in output)
{
foreach (T data in Record)
{
objectOut.Add(data);
}
}
}
catch (Exception ex)
{
//Log exception here
Console.Write(ex.Message);
}
return objectOut;
}
}
call to process
{
string fname = "../../Names.json"; // <- your json file path
// for alternate types replace string with custom class below
List<string> jsonFile = JsonObj.DeSerializeObject<string>(fname);
}
or this call to process
{
string fname = "../../Names.json"; // <- your json file path
// for alternate types replace string with custom class below
List<string> jsonFile = new List<string>();
jsonFile.AddRange(JsonObj.DeSerializeObject<string>(fname));
}

If you're doing this in the context of a asp.Net Core API action, the conversion to Json is done implicitly.
[HttpGet]
public ActionResult Get()
{
return Ok(TheList);
}

using System;
using System.Text.Json;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
List<ErrorDetail> aList = new List<ErrorDetail>();
ErrorDetail a = new ErrorDetail{ ColumnName="aaa", ErrorText="abbbb"};
ErrorDetail c = new ErrorDetail{ ColumnName="ccc", ErrorText="cccc"};
ErrorDetail b = new ErrorDetail{ ColumnName="ccc", ErrorText="cccc"};
aList.Add(a);
aList.Add(b);
aList.Add(c);
var json = JsonSerializer.Serialize(aList);
Console.WriteLine(json);
}
public class ErrorDetail
{
public string ColumnName { get; set; }
public string ErrorText { get; set; }
}
}

I tried the other answers here to serialize parameters for a POST request, but my backend did not like the fact that I was sending up a string version of my array. I did not want to have to always check if the type of a parameter is a string and convert it to an array.
I'm using Json.NET (which is now built into C#), and I convert my List to an array and let the converter handle the rest.
public class MyObjectInJson
{
public long ID;
public OtherObject[] array;
}
You can convert your List into an array using list.ToArray();
And then finally using the JsonConvert, you can turn the entire object into a string:
string jsonString = JsonConvert.SerializeObject(objectInJson);
Hope this helps someone else.

XmlSerializer output xml types

I am using the XmlSerializer, and was wondering if there is any way, using overrides or something to that effect to get the XmlSerializer to output the types of some nodes.
My problem is that I have serialized a byte array.
class MyClass {
public string Name { get; set; }
public byte[] Bytes { get; set; }
}
I am consuming the xml in a generic service.
The service collects the xml as .
<MyClass>
<Name>Test</Name>
<Bytes>U2NhcnkgQnVnZ2Vy</Bytes>
</MyClass>
Is there any way to either generate an xsd at runtime, or somehow output something like this.
I cannot change the class I am serializing, but I can apply overrides to the serializer or in some other way control the serialization.
<Bytes xsi:type='BinaryOfSomeKind'>BlahBlah</Bytes>
I need to know that the data is binary somehow.
Thanks
Craig.

If your class is supplied by a third party then you know your properties and property types and you can deduce your XML and XSD from it. You can create your XSD manually or with the help of a XML tool for example XMLSpy (not free BTW) or XMLFox which is free of charge.

If you know the xml is going to be in that format that you put in the question and you have your class ready you can decorate it as such for it to be deserialized.
The Deserialization class:
[XmlTypeAttribute]
[XmlRootAttribute("MyClass")]
public class MyClass
{
[XmlElementAttribute("Name")]
public string Name { get; set; }
[XmlElementAttribute("Bytes")]
public byte[] Bytes { get; set; }
}
The Deserialzation Method
public static object Deserialize(string xml)
{
var deserializer = new System.Xml.Serialization.XmlSerializer(typeof(MyClass));
using (var reader = XmlReader.Create(new StringReader(xml)))
{
return (MyClass)deserializer.Deserialize(reader);
}
}
The Main Method
static void Main(string[] args)
{
string xml = #"<MyClass>
<Name>Test</Name>
<Bytes>U2NhcnkgQnVnZ2Vy</Bytes>
</MyClass>";
MyClass obj = (MyClass)Deserialize(xml);
Console.ReadKey();
}
Make sure to add the following using statements:
using System.Xml.Serialization;
using System.Xml;
It deserialized it into an obj with "Test" as the byte array.
If you generate the XSD at run time, then theres no way you can know what properties there are and it would be down to using reflection to test for specific properties, and then subsequently finding out what types they may be, is this what your after?