I have an application with plugin architecture. The plugins have their own data containers, which all inherit from the base interface IPluginDataStorage. The DataStorage object, which is serialized to preserve state, contains a list of these subclasses along with other data. The DynamicType flag is set to true, as it's not known until run time which plugins are in use.
[Serializable]
[ProtoContract]
public class DataStorage
{
[ProtoMember(200, DynamicType = true)]
public List<IPluginDataContainer> PluginDataStorage { get; set; }
When serializing this setup works okay, but I'm having problems deserializing the list reliably. If I try to deserialize the object without having access to all those plugins that were used when it was serialized, I, naturally, get an exception about a missing type.
Unable to resolve type: NOSMemoryConsumptionPlugin.NOSMemoryConsumptionData, NOSMemoryConsumptionPlugin, Version=1.2.0.17249, Culture=neutral, PublicKeyToken=null (you can use the TypeModel.DynamicTypeFormatting event to provide a custom mapping)
The exception gives a hint that I could provide the format through the event, but I don't see how that could be helpful as the problem is that I don't have that type available. What I'd like to do in these cases is to completely ignore deserializing that object. The list item could even default to a dummy instance of the base class in these cases. But how to do this?
That (choosing to skip or default) is a fascinating use-case that I don't think I have considered fully; however, you can probably do this yourself, via:
public class NilContainer : IPluginDataContainer {}
and then subscribe to the DynamicTypeFormatting event; if you don't recognise the type, supply typeof(NilContainer).
i.e.
RuntimeTypeModel.Default.DynamicTypeFormatting += (sender, args) =>
{
Type type;
if(!yourTypeMap.TryGetValue(args.FormattedName, out type))
{
type = typeof (NilContainer);
}
args.Type = type;
};
(completely untested)
Related
Let's say I have this little json snippet:
{
"Type": "Bar",
"BarOnly": "This is a string readable when deserialized to the Bar class only, as declared in my type key"
}
I also have these three classes:
public class Base
{
public enum SampleEnum
{
Bar,
Baz,
}
public SampleEnum Type
{
get;
set;
}
}
public class Bar : Base
{
public string BarOnly
{
get;
set;
}
}
public class Baz : Base
{
public string BazOnly
{
get;
set;
}
}
Based on the Type property in the json snippet, I'd like to have it deserialize to either Bar or Baz.
My first idea was to first deserialize it to the Base class, and then use its type and a switch statement to deserialize the JSON again to its respective class. (Using Newtonsoft.Json)
var type = JsonConvert.DeserializeObject<Base>(json).Type;
string message = "";
switch (type)
{
case (Base.SampleEnum.Bar):
message = JsonConvert.DeserializeObject<Bar>(json).BarOnly;
break;
case (Base.SampleEnum.Baz):
message = JsonConvert.DeserializeObject<Baz>(json).BazOnly;
break;
}
Console.WriteLine(message);
Needless to say that this process is extremely redundant, tedious and, since the switch statement is hard-coded, not very "dynamic" at all.
Another idea was to use a generic class as the base class instead and passing in the type of the actual class it should deserialize to, but then I end up with the same switch statement to figure out what that class should be.
Since you can't map enums to class types, I also thought about using a Dictionary to map the possible enum values to their class counterparts; this still makes the mapping process hard-coded though.
Is there any way I can dynamically get the corresponding class to deserialize to based on the type property of the json object?
EDIT: There seems to be some confusion about how this is supposed to be used and how the data is fetched; let me provide some background information.
I'm iterating through a directory with a lot of different spreadsheet files, mostly CSVs and XML files. Each of these feeds have a "meta file", describing how to process their content. This includes checksums, delimiters and other information. They also declare of what type their parent file is (CSV, XML etc). Hence, they share a lot of common properties (like the Base class in my example), but also have their own set of properties. They derive from an abstract class that requires them to implement a function that returns an instance of the corresponding feed processing class, initialized with values directly from within the meta class. I hope this makes sense.
#OguzOzgul commenting is correct. I've done this countless of times for objects that are composed with interfaces that need to be serialized and deserialized.
See TypeNameHandling for Newtonsoft:
https://www.newtonsoft.com/json/help/html/SerializeTypeNameHandling.htm
Your json file will look ever so slightly different:
{
"$type": "SomeNamespace.Bar",
"BarOnly": "This is a string readable when deserialized to the Bar class only, as declared in my type key"
}
If you use
new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.All
}
During serialization, it will add the full type name of all objects to make sure newtonsoft knows what their type is during deserialization (given you use the same settings then). That way you do not have to write your own custom code for type detection.
How come when I send a message to RabbitMQ through rebus that it is adding extra object data like below, primarily the $type.
{"$type":"ThreeSquared.VTGPAM.Objects.Wagon, ThreeSquared.VTGPAM.Objects","WagonId":"a98a06ab-33b9-4a11-9de2-df0b8787b713","WamosId":12324,"Description":"test","YearBuilt":1982,"Token":"3cce443c-249f-4fd2-9882-5830fb308b6b"}
We have a client that will just be using the Java RabbitMQ library with no rebus. This approach I believe we just send the JSON without the type declarations. This therefore doesn't work when I try and read in simple JSON object. How can we make it work so that it doesn't define the $type in the message?
It's simply because Rebus by default uses Newtonsoft JSON.NET with TypeNameHandling.All, which means that the $type field is included in every serialized object containing the full .NET type name of the type serialized.
The benefit is that you can serialize almost anything, even though it may contain instances referenced by (possibly abstract) supertypes, and even by interfaces.
E.g. this command message type
public class ProcessFile
{
public ProcessFile(string filePath, IEnumerable<IFileProcessingTask> tasks)
{
FilePath = filePath;
Tasks = tasks;
}
public string FilePath { get; }
public IReadOnlyCollection<IFileProcessingTask> Tasks { get; }
}
could contain arbitrary implementations of IFileProcessingTask, e.g. something like
public class GZipFileProcessingTask : IFileProcessingTask
{
// ...
}
as long as the recipient can find the type by looking it up via the value of the $type field.
If you want to process this type of message on another platform, you can simply make it ignore the $type field of every object. This may be easy/hard/impossible, depending on how flexible your JSON serializer is.
Another option is to simply replace Rebus' serializer with your own implementation by doing this
Configure.With(...)
.(...)
.Serialization(s => s.UseCustomJsonSerialization())
.Start();
where UseCustomJsonSerialization is an extension method that you implement like this:
public static class RebusConfigEx
{
public static void UseCustomJsonSerialization(this StandardConfigurer<ISerializer> configurer)
{
configurer.Register(c => new YourCustomJsonSerializer());
}
}
and then all there is left to do is to create the class YourCustomJsonSerializer as an implementation of ISerializer.
I am attempting to save/load a class to an xml file that contains generic types using a DataContractSerializer. I have the save working, but have realized I can't load it because I don't have the list of knownTypes for the deserializer.
Is there a way of serializing/deserializing this class that would allow me to deserialize it without referencing any of the stored types directly?
Here is my SessionVariables class that I am trying to save/load:
[DataContract]
public class SessionVariables
{
[DataMember]
private Dictionary<Type, ISessionVariables> _sessionVariables = new Dictionary<Type, ISessionVariables>();
private object _syncLock = new object();
public T Get<T>()
where T : ISessionVariables, new()
{
lock (_syncLock)
{
ISessionVariables vars = null;
if (_sessionVariables.TryGetValue(typeof(T), out vars))
return (T)vars;
vars = new T();
_sessionVariables.Add(typeof(T), vars);
return (T)vars;
}
}
public IList<Type> GetKnownTypes()
{
IList<Type> knownTypes = new List<Type>();
knownTypes.Add(this.GetType().GetType()); // adds System.RuntimeType
foreach (Type t in _sessionVariables.Keys)
{
if (!knownTypes.Contains(t))
knownTypes.Add(t);
}
return knownTypes;
}
}
The different modules of the application extend the ISessionVariables interface to create their own set of session variables, like this:
[DataContract]
public class ModuleASessionVariables : ISessionVariables
{
[DataMember]
public string ModuleA_Property1{ get; set; }
[DataMember]
public string ModuleA_Property2 { get; set; }
}
[DataContract]
public class ModuleBSessionVariables : ISessionVariables
{
[DataMember]
public string ModuleB_Property1{ get; set; }
[DataMember]
public string ModuleB_Property2 { get; set; }
}
And a singleton instance of the SessionVariables class is used to access session variables, like this:
singletonSessionVariables.Get<ModuleASessionVariables>().ModuleA_Property1
singletonSessionVariables.Get<ModuleBSessionVariables>().ModuleB_Property2
I got the save working like this:
using (FileStream writer = new FileStream(#"C:\test.txt", FileMode.Create))
{
DataContractSerializer dcs = new DataContractSerializer(typeof(SessionVariables), singletonSessionVariables.GetKnownTypes());
dcs.WriteObject(writer, singletonSessionVariables);
writer.Close();
}
However this method does not work to deserialize the class because I don't know it's known types.
Can I serialize and deserialize generic types when I don't have direct library references to any of the types used? And if so, how?
The problem here is that you aren't just wanting to serialize data, but you also want to serialize data about your data, i.e., (cue the dramatic chipmunk) metadata.
That metadata, in this case, are the types of the models that held the data originally. Normally, this isn't an issue, but as you've discovered if you're taking advantage of polymorphism in your design, your single collection may contain two or more different types, each of which needs to be deserialized to their original type.
This is usually accomplished by saving this Type metadata to the serialized result. Different serialization methods does this in different ways. Xaml serialization uses xml namespaces associated with .net namespaces, then names the elements after the original type name. Json.net accomplishes this via a specific named value saved to the json object.
The default DataContractSerializer is not Type aware. Therefore you need to replace it with a version that understands the .NET Type system and can serialize/deserialize Type metadata to the resulting xml. Luckily, one already exists in the framework, the NetDataContractSerializer.
And that's how you pad a link-only answer. The Aristocrats.
You could accomplish this using a custom DataContractResolver. This allows you to plug into the deserialization pipeline and provide a type to deserialize into based upon the type/namespace that is found in the serialized graph.
Here's a good article on it:
http://blogs.msdn.com/b/carlosfigueira/archive/2011/09/21/wcf-extensibility-data-contract-resolver.aspx
IDesign has an implementation of a resolver that can be used for dynamic discovery of types on their site: http://idesign.net/Downloads/GetDownload/1848 (you will probably have to make some modifications to handle generics)
I have a curious case of this serialization issue - which has been asked many times on this site and I have gone through a few of these questions and tried the usual items to no avail:
Add [XmlInclude] to the class throwing the error
Remove namespaces
Add a different namespace to each class
To explain further, I have provided a simplified version of my code below. Essentially I am using a WebServiceHost object to run a RESTful service and one of my endpoints returns an object serialized as XML (I have annotated the object with [DataContract] and [DataMember] attributes). This object contains a SerializableDictionary<string, object> (here) where the value has been typed as object. I believe this is why it is failing:
Works fine when the value is assigned a primitive
When I assign a custom object to the KV pair V, I get the unexpected type exception probably because the Serializer does not know how to serilaize the object / some sort of namespacing issue
Obviously, I am unable to annotate Object.cs with [XmlInclude] and because it is a service and I am not myself serializing I cannot using something like
new Serializer(typeof(...), new Type[] { ... }}
Any idea's of what I can do? I thought about not typing the dict value as object and rtaher comething more concrete but the problem is that this value can take primitives or cusotm types. Some code to explain the above:
Edit: Updated the code below to make it more clear
[DataContract]
public class ResponseObject
{
[DataMember(Name = "data")]
public SerializableDictionary<string, object> Data { get;set; }
public ResponseObject()
{
Data = new SerializableDictionary<string, object>();
}
}
...
var d1 = new ResponseObject();
d1.Data.Add("some key", "some value"); //WORKS AND SERIALIZES PERFECLTY
var d2 = new ResponseObject();
d2.Data.Add("some other key", new SomeOtherObjecT());
var d3 = new ResponseObject();
d3.Data.Add("another key", d2); //THIS THROWS THE UNEXPECTED TYPE ERROR WHEN SEIRLAIZING SomeOtherObject
Edit: The error is thrown in SerializableDictionary where it is attempting to serialize an object of type ResponseObject. The two are in seperate projects - if that is significant?
Normally, you should add an [XmlInclude] to the ResponseObject class. In this case, it doesn't work because of the SerializableDictionary that you're using. That class creates another XmlSerializer in its implementation, and therefore it doesn't care about your [XmlInclude]'s. Basically it just cannot handle your use case. You should switch from the XmlSerializer to the DataContractSerializer which handles the Dictionary class and supports the [KnownType] attribute to register additional types: http://pastebin.com/vGLSaxHF . Also note that it's pointless to add [DataContract] and [DataMember] attributes in your current case because the XmlSerializer ignores those attributes, they are used by the DataContractSerializer only. Or if you're not sure how to change your serializer (I know I'm not) then you should either not be using a Dictionary or change the SerializableDictionary implementation to handle the dynamic object types that you want to use (find every line where it creates a new XmlSerializer). Or, as an alternative, define a base class for all your objects that you will ever put into the dictionary and do it like this:
[XmlInclude(typeof(Class1), XmlInclude(typeof(Class2)), etc]
public class AbstractBase { }
public class Class1 : AbstractBase { ... }
public class Class2 : AbstractBase { ... }
public class BigClass {
public SerializableDictionary<string, AbstractBase> Dictionary { get; set; }
}
This way, when the SerializableDictionary creates its own XmlSerializer, it will recognize the AbstractBase and from there, all of its descendants.
I have about 20 classes for different messages and this number are growing. Each class has a unique ID, so I can transform the class on a byte[] with my own method of serialization and then transform a byte[] again on my class with this uniqueID.
All my messages are children of a BaseMessage class that already implements the uniqueID generation correctly.
What I want to do is direct find the class of a respective ID without using a Enum to compare.
My problem with the Enum is that Enums are not auto updated with my new IDs every time I create a new message class.
There a way to combine Attributes and Assembly to do this, Like discovering all children of BaseClass and then call for a CustomAtributte?
Thank you!
You're on the right path - that does sound like a good way to handle it. You'll want to store the unique ID of the type alongside the serialized value, so you can read the ID before deserialization, to direct the deserializer to the correct type. You could also just store the fully qualified type name instead of using an ID, but this is a fine approach too.
The attribute is simple enough to create:
class MessageAttribute : Attribute
{
public Guid ID; //assuming you want to use a GUID, could be anything
}
And using it is also simple:
[Message(ID = new Guid("..."))]
class SubMessage : BaseMessage
{
}
You can load all the types in a given assembly and iterate over them very quickly:
List<Type> messageTypes = new List<Type>();
//get the assembly containing our types
Assembly messageAssembly = Assembly.Load(...);
//get all the types in the assembly
Type[] types = messageAssembly.GetTypes();
foreach(Type type in types)
{
//make sure we inherit from BaseMessage
if(type.IsAssignableFrom(typeof(BaseMessage))
{
//check to see if the inherited type has a MessageAttribute
object[] attribs = type.GetCustomAttribtues(typeof(MessageAttribute), true);
if(attribs.Length > 0)
{
messageTypes.Add(type);
}
}
}