I have a host of classes including the normal designs of object hierarchies and interfaces, base classes, etc. from a project where I cannot modify any code. I have another payload class in my project which uses composition to encapsulate information from the other classes and contain properties in the payload class whose types are the classes from the other project.
Now I have a need to be able to create an instance of the payload class containing instances of those other classes and serialize it to Base64 string for transmission.
Issue is since I cannot touch the code for the other classes, I cannot add serialization attributes (for .NET binary formatter/protobuf-net). I was also trying to look at using protobuf-net without attributes, but since the object hierarchy is too deep, it seemed to be too complicated to create.
Can somebody tell me a better choice to go ahead with the serialization/deserialization part without modifying the existing code.
Sample code to illustrate the requirement is shown below:
void Main()
{
var b = new B();
b.SetStatus("This is B");
var c = new C { Name = "C", Value = 100};
var payload = new Payload(b, c);
var serializedData = SerializeToString<Payload>(payload);
serializedData.Dump();
}
private static TData DeserializeFromString<TData>(string settings)
{
byte[] b = Convert.FromBase64String(settings);
using (var stream = new MemoryStream(b))
{
var formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
return (TData)formatter.Deserialize(stream);
}
}
private static string SerializeToString<TData>(TData settings)
{
using (var stream = new MemoryStream())
{
var formatter = new BinaryFormatter();
formatter.Serialize(stream, settings);
stream.Flush();
stream.Position = 0;
return Convert.ToBase64String(stream.ToArray());
}
}
public class A
{
public string Id { get; set; }
public string StatusMsg {get;protected set;}
public virtual void SetStatus(string msg)
{
StatusMsg = msg;
}
}
public class B : A
{
public B()
{
Id = new Guid().ToString();
}
public override void SetStatus(string msg)
{
base.SetStatus(msg);
}
}
public class C
{
public string Name
{
get;
set;
}
public Int32 Value
{
get;
set;
}
}
public class Payload
{
public B PropertyB { get; set; }
public C PropertyC { get; set; }
public Payload(B b, C c)
{
this.PropertyB = b;
this.PropertyC = c;
}
}
Without adding configuration attributes, you have a few options:
use a serialize that won't care: XmlSerializer or Json.NET might work if you're lucky
accept the runtime config work of something like protobuf-net or one of the others; it probably isn't as much work as you expect (heck, drop me an email with real code and I might be able to do it)
wrote the serialization entirely manually
write a DTO layer - a basic model with attributes etc that works well with your chosen serializer - and either write code to map between the two models, or use a tool like auto-mapper
Personally, when serialization configuration gets tricky, my default option is "write a DTO model".
I don't understand why protobuf would not work but you can do the serialization manually if you don't mind.
In that case, create a BinaryWriter and then write everything you need to be able to deserialize it back again using a BinaryReader. Doing it this way you will get a very compact representation tailored for you specific needs which is also very fast. But it's a bit more work also. General purpose serialization can be verbose and slow but is almost always preferred in any case.
But I still don't get why using any existing attribute-less serialization method wouldn't work. You should probably start by really understand why it doesn't work with for instance protobuf or JSON.NET. You mentioned "too deep hierarchy". How deep is that?
Related
What is the best approach when I do not know at runtime which types are de/serialized with protobuf?
Currently I am playing with the idea to extend the RuntimeTypeModel in the type initializers of the types which are candidates for serialization which seems to work pretty well for serialiation. But when deserializing in a different process I would need to load the same type model from somewhere which was used to serialize the types. Is it possible to serialize the RuntimeTypeModel to disk to reuse it later when the serialized data is read again from disk? Ideally I would put the model into the serialized stream as well to have a full self describing object model. Or would I need to record the steps and put this data in front of my serialized stream?
One could create a header which contains the offset to the real data and the runtime type model and the length which would be pretty nice. Or is there a better approach how to deal with a plug in architecture where at serialization time I have all types registered but during deseralization I might stil need to load some types from their respective assemblies because the code was not yet touched?
using ProtoBuf;
using ProtoBuf.Meta;
using System.Collections.Generic;
using System.IO;
namespace protobuf
{
[ProtoContract]
public interface IAbstraction
{
[ProtoMember(1)]
string Name { get; set; }
}
[ProtoContract]
public class Base : IAbstraction
{
static Base()
{
ProtobufTypeModels.MainModel.Add(typeof(IAbstraction), true).AddSubType(101, typeof(Base));
}
[ProtoMember(1)]
public string Name { get; set; }
[ProtoMember(2, AsReference =true)]
public List<IAbstraction> Instances = new List<IAbstraction>();
}
[ProtoContract]
public class Next : Base
{
static Next()
{
ProtobufTypeModels.MainModel.Add(typeof(IAbstraction), true).AddSubType(100, typeof(Next));
}
[ProtoMember(1)]
public string NextName { get; set; }
}
public static class ProtobufTypeModels
{
public static readonly RuntimeTypeModel MainModel = TypeModel.Create();
}
class Program
{
static void Main(string[] args)
{
Base b = new Base { Name = "Alois" };
b.Instances.Add(new Next { Name = "Base", NextName = "Christian" });
b.Instances.Add(new Base { Name = "SecondBase", Instances = b.Instances });
var mem = new MemoryStream();
ProtobufTypeModels.MainModel.Serialize(mem, b);
mem.Position = 0;
var deser = (Base) ProtobufTypeModels.MainModel.Deserialize(mem, null, typeof(Base));
}
}
}
Random thought: you could use .Compile(serializerName,dllPath) after you've finished the initialization and write the baked serializer to disk; then you can reference it, use new SerializerName() to create the instance, and use the .Serialize etc methods from there. The dll will never change. This also means it never has to process any metadata ever again; no reflection, no IL emit, etc.
Other than that: we could possibly do something more gentle in terms of storing the configuration, but: protobuf-net doesn't currently add anything directly to support it, and it would probably be more relevant for you to have your own bespoke configuration data that you simply consume at startup.
I have the following Console application:
using System;
using System.IO;
using System.Xml.Serialization;
using Newtonsoft.Json;
namespace OutputApp
{
public class Foo
{
public object Value1 { get; set; }
public string Value2 { get; set; }
}
public class Bar
{
public int Arg1 { get; set; }
public double Arg2 { get; set; }
}
class Program
{
public static Foo CreateFooBar()
{
return new Foo
{
Value1 = new Bar
{
Arg1 = 123,
Arg2 = 99.9
},
Value2 = "Test"
};
}
public static string SerializeXml(object obj)
{
using (var stream = new MemoryStream())
{
using (var reader = new StreamReader(stream))
{
var serializer = new XmlSerializer(obj.GetType());
serializer.Serialize(stream, obj);
stream.Position = 0;
return reader.ReadToEnd();
}
}
}
static void Main(string[] args)
{
var fooBar = CreateFooBar();
// Using Newtonsoft.Json
var json = JsonConvert.SerializeObject(fooBar, Formatting.Indented);
var xnode = JsonConvert.DeserializeXNode(json, "RootElement");
var xml = xnode.ToString();
// Using XmlSerializer, throws InvalidOperationException
var badXml = SerializeXml(fooBar);
Console.ReadLine();
}
}
}
I have two classes. Class Foo and class Bar. Class Foo has a property of type object. This is a requirement, because it is a contract which can hold a variety of objects and therefore I cannot set the property to a concrete type or a generic.
Now I compose a dummy fooBar object using the CreateFooBar() method. After that I first serialize it into JSON, which works wonderfully with Json.Net.
Then I use Json.Net's XML converter method to convert the json string into an XNode object. It works great as well.
The output of both is the following:
{
"Value1": {
"Arg1": 123,
"Arg2": 99.9
},
"Value2": "Test"
}
<RootElement>
<Value1>
<Arg1>123</Arg1>
<Arg2>99.9</Arg2>
</Value1>
<Value2>Test</Value2>
</RootElement>
Now while this works, it is certainly not very nice, because I have to serialize into json only to serialize it into xml afterwards. I would like to serialize directly into xml.
When I use the XmlSerializer to do this I get the infamous InvalidOperationExceptoin, because I did not decorate my classes with the XmlInclude attribute or did one of the other workarounds.
InvalidOperationException
The type OutputApp.Bar was not expected. Use the XmlInclude or
SoapInclude attribute to specify types that are not known statically.
None of the workarounds for the XmlSerializer is a good solution IMHO and I don't see the need for it as it is perfectly feasible to serialize an object into XML without crappy attributes.
Does anyone know a good Xml serializer in .NET which can do this or is there a plan to add this feature to Json.Net?
Any ideas?
Update1
I am not opposed to use attributes, but it needs to make sense. What I don't like about the XmlInclude attribute is that it forces me into circular dependencies. Say I have assembly A which defines a base class, and assembly B which implements derived classes. Now the way the XmlInclude attribute works is that I'd have to decorate the base class in Assembly A with the type name of the child class from assembly B. This would mean I have a circular dependency and is a no go!
Update2
I shall clarify that I am not looking for a solution to re-factor my console application to make it work with the XmlSerializer, I am looking for a way to XML serialize what I have there.
There was a comment below which mentions that using object as a data type is poor design. Whether this is true or not, this is a whole other discussion. The point is that there is no reason why it shouldn't be able to serialize into XML and I am curious to find such a solution.
Personally I find creating a "marker" interface a dirty design. It abusing an interface to workaround the incapabilities of one single .NET class (XmlSerializer). If I would ever swap the serialization library for something else, then the whole marker interface would be redundant clutter. I don't want to couple my classes to one serializer.
I am looking for an elegant solution (if there is one)?
You don't need to pollute your models with XmlInclude attributes. You could explicitly indicate all known classes to the XmlSerializer's constructor:
var serializer = new XmlSerializer(obj.GetType(), new[] { typeof(Bar) });
Also using object as base class seems like a crappy approach. At least define a marker interface:
public interface IMarker
{
}
that your Bar's will implement:
public class Bar : IMarker
{
public int Arg1 { get; set; }
public double Arg2 { get; set; }
}
and then then specialize the Value1 property of your Foo class to this marker instead of making it like the most universal type in the universe (it can't be):
public class Foo
{
public IMarker Value1 { get; set; }
public string Value2 { get; set; }
}
Coz now it's pretty trivial to get all loaded types at runtime in all referenced assemblies that are implementing the marker interface and passing them to the XmlSerializer constructor:
var type = typeof(IMarker);
var types = AppDomain.CurrentDomain.GetAssemblies()
.SelectMany(s => s.GetTypes())
.Where(p != type)
.Where(p => type.IsAssignableFrom(p))
.ToArray();
var serializer = new XmlSerializer(obj.GetType(), types);
Now you've got a pretty capable XmlSerializer that will know how to properly serialize all types implementing your marker interface. You've achieved almost the same functionality as JSON.NET. And don't forget that this XmlSerializaer instantiation should reside in your Composition Root project which knows about all loaded types.
And once again using object is a poor design decision.
I wrote some abstract class that support quick serialize and deserialize.
I add StructLayout to the implementation class and i got crash ( without the StructLayout its working with no crash )
I have 2 question
Is there is any better fest way to do the Serialize ( on this i do serialization in TotalMilliseconds = 0.4834 and using the standard way of .net i takes TotalMilliseconds = 2.120 ) ?
Why the 'StructLayout' crash my code ( exception on the runtime at the start of the application running
{"Could not load type 'WindowsFormsApplication4.Element1' from assembly 'WindowsFormsApplication4, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' because the format is invalid.":"WindowsFormsApplication4.Element1"}
The code
public abstract class SerializeElement
{
public abstract void Serialize(BinaryWriter bw);
public abstract void DeSerialize(BinaryReader br);
public sealed byte[] Serialize()
{
byte[] retVal = null;
using(MemoryStream ms = new MemoryStream())
{
using(BinaryWriter bw = new BinaryWriter(ms))
{
Serialize(bw);
retVal = ms.GetBuffer();
}
}
return retVal;
}
public sealed void DeSerialize()
{
byte[] retVal = null;
using(MemoryStream ms = new MemoryStream())
{
using(BinaryReader br = new BinaryReader(ms))
{
DeSerialize(br);
}
}
}
}
[StructLayout(LayoutKind.Sequential, Pack=4)]
public class Element1 : SerializeElement
{
public int Var1 { get; set; }
public int Var2 { get; set; }
public override void Serialize(BinaryWriter bw)
{
bw.Write(Var1);
bw.Write(Var2);
}
public override void DeSerialize(BinaryReader br)
{
Var1 = br.ReadInt32();
Var2 = br.ReadInt32();
}
}
static void Main()
{
Element1 e = new Element1()
{
Var1 = 1,
Var2 = 2
};
var tt = e.Serialize();
OK, after a little testing, I found out that .Net doesn't like the fact that Element1 inherits from SerializeElement while having the StructLayout attribute applied.
When I commented out inheriting from SerializeElement, the program ran fine. So, I suggest that you create a Data Transfer Object that encapsulates the properties used by Element1 and uses StructLayout attribute. Then, provide a conversion to a class that inherits SerializeElement so that you can serialize and deserialize that value.
You will need to make sure that the output is formatted correctly, but that shouldn't be too much of a problem.
In terms of BinaryWriter being the fastest solution, I would assume so. BinaryWriter is a simple abstraction over a Stream that just provides utilities for converting values to/from int/float/double/etc. values. But, the important part is knowing when you need the fastest solution. Time and time again, we(programmers) find out that sometimes the fastest solution isn't the best one. Often, the trade off of speed for development pace and quality is worth it.
With that said, I would recommend (as would others) that you try using protobuf-net. It is quite fast, and (in my opinion) the benefits of a general purpose serializer far outweigh having to write custom serializers for every single one of your classes.
Sometimes the answer is just a click away ... the StructLayout definition don't give the inherited option
I'm using protobuf-net version 2.0.0.640 to serialize some data as shown below.
[ProtoContract(ImplicitFields = ImplicitFields.AllFields)]
public interface ITestMessage
{
}
[ProtoContract(ImplicitFields = ImplicitFields.AllFields)]
public class MyOrder : ITestMessage
{
public int Amount { get; set; }
}
[ProtoContract(ImplicitFields = ImplicitFields.AllFields)]
public class MyOrderWrapper
{
public MyOrder Order { get; set; }
}
[TestMethod]
public void TestOrderSerialize()
{
var order = new MyOrder() {Amount = 10};
var orderWrapper = new MyOrderWrapper() { Order = order };
using (var file = File.Create("C:\\temp\\order.bin"))
{
Serializer.Serialize<MyOrderWrapper>(file, orderWrapper);
}
}
Now, If I declare an inheritance dependency between 'ITestMessage' & 'MyOrder' via code using:
RuntimeTypeModel.Default[typeof(ITestMessage)].AddSubType(2, typeof(MyOrder));
I get the following error when trying to deserialize my prevously saved data.
"No parameterless constructor found for ITestMessage".
[TestMethod]
public void TestOrderDeserialize()
{
RuntimeTypeModel.Default[typeof(ITestMessage)].AddSubType(2, typeof(MyOrder));
MyOrderWrapper orderWrapper;
using (var file = File.OpenRead("C:\\temp\\order.bin"))
{
orderWrapper = Serializer.Deserialize<MyOrderWrapper>(file);
}
}
Can someone please explain why this would happen when 'MyOrderWrapper' does not reference anything higher than 'MyOrder' in the inheritance hirarchy.
Also, why it works when I explictly include '[ProtoInclude(2, typeof(MyOrder))]' on 'ITestMessage'
Thanks
Basically, that is a breaking change as far as the serializer is concerned - at the wire layer, neither "classes" nor "interfaces" exist, so in terms of storage, this is akin to changing the base-type of the class; when serializing, the root type was MyOrder - and during deserialization the root type is ITestMessage. This will not make it happy.
Basically: you can't do that.
Suppose I have class CarResource,
class RaceCarResource : public CarResource,
and class SuperDuperUltraRaceCarResource : public RaceCarResource.
I want to be able to load their data using a single method LoadFromXML.
How would I go about getting the CarResource:LoadFromXML to load it's data,
RaceCarResource to call CarResource:LoadFromXML and then load it's own additional data, etc. ?
If I use XmlTextReader I only know how to parse the entire file in one go,
not how to use it so first CarResource:LoadFromXML can do its thing, then RaceCarResource, etc.
I hope it's at least a little bit clear what I mean :)
public class CarResource
{
public virtual void LoadFromXML(String xmlData)
{
...
}
}
public class RaceCarResource : CarResource
{
public override void LoadFromXML(String xmlData)
{
base.LoadFromXML(xmlData);
...
}
}
...and so on. The new keyword will hide the inheritted method but still allow it to be call-able from the child class.
As for actually parsing the XML, you have a couple of options. My first suggestion would be to read the entire XML file in to memory...and then use LINQ to XML to parse through and populate your classes. You could also try the XmlSerializer (LINQ to XML is easier to implement, but as the size of your code-base grows, Xml Serialization can make maintenance easier).
You could also use XML Serialization depending on the structure of your XML file to load from. It's possible to override the load method (and then override in subsequent classes) to load specific information - or just use attributes. See: http://msdn.microsoft.com/en-us/library/ms950721.aspx
You have a couple of options.
You can use Linq to XML to query the child entities and pass those nodes to your other classes. This is probably the most efficient way of doing it.
You could use an xmlnavigator, again only passing the appropriate child nodes...
see: Implementing my own XPathNavigator in C#
You could simply use xml serialization (XmlSerialize XmlDeserialize), see C# - How to xml deserialize object itself?
In order to use XML de-serialization, the instance method makes the current object effectively 'immutable', but I would suggest something like this:
public class CarResource
{
public CarResource LoadNewFromXML(string xml)
{
XmlSerializer ser = new XmlSerializer(this.GetType());
object o = null;
using (MemoryStream ms = new MemoryStream(Encoding.ASCII.GetBytes(xml)))
{
o = ser.Deserialize(ms);
}
return o as CarResource;
}
}
public class RaceCarResource : CarResource
{
}
public class SuperRaceCarResource : RaceCarResource
{
}
Your calling code then looks like:
RaceCarResource car = new RaceCarResource();
car = car.LoadNewFromXML("<RaceCarResource/>") as RaceCarResource;
SuperRaceCarResource sc = new SuperRaceCarResource();
sc = sc.LoadNewFromXML("<SuperRaceCarResource/>") as SuperRaceCarResource;
If your XML is not compatible with the .net XML serialisation, then the easiest way is to create a factory which detects which type of resource the XML represents, then handles that appropriately. If you want to put the parsing into your objects, then use a virtual method to parse the internals after creating the object:
class CarResource
{
public string Color { get; private set; }
internal virtual void ReadFrom(XmlReader xml)
{
this.Color = xml.GetAttribute("colour");
}
}
class RaceCarResource : CarResource
{
public string Sponsor { get; private set; }
internal override void ReadFrom(XmlReader xml)
{
base.ReadFrom(xml);
this.Sponsor = xml.GetAttribute("name-on-adverts");
}
}
class SuperDuperUltraRaceCarResource : RaceCarResource
{
public string Super { get; private set; }
internal override void ReadFrom(XmlReader xml)
{
base.ReadFrom(xml);
this.Super = xml.GetAttribute("soup");
}
}
class CarResourceFactory
{
public CarResource Read(XmlReader xml)
{
CarResource car;
switch (xml.LocalName)
{
case "ordinary-car": car = new CarResource(); break;
case "racecar": car = new RaceCarResource(); break;
case "super_duper": car = new SuperDuperUltraRaceCarResource(); break;
default: throw new XmlException();
}
XmlReader sub = xml.ReadSubtree();
car.ReadFrom(sub);
sub.Close();
return car;
}
}
This works OK if the XML for a sub-type has any child elements appended strictly after or before the content for the super-type. Otherwise you have to do more work to reuse the super-type's serialisation, breaking it up into smaller methods (eg the base has methods to load the number of wheels, doors, engine size; the race car calls LoadDoorData, LoadAeroFoilData, LoadWheelData if the XML for the race car has the aerofoil data in between the door and wheel data. For formats with no logical ordering imposed (XMI, RDF) you have to inspect the local name to decide which specialised method to call, which gets a bit messy if you want to combine it with virtual methods. In that case, it's better to use a separate serialisation helper.
Other mechanisms can be used in the factory if the set of types to be created is not fixed to a few types.