We Keep Coding

How to to serialize a class with a private dictionary member into a file?

I am trying to write functions that save a type, the Dispatcher type shown below, into a file, and then reload it later. I wrote that following functions, but they are not working. I get an exception:
Exception thrown: 'System.InvalidOperationException' in System.Xml.dll
Elsewhere I read that maybe because the member in the class is private then I should use BinaryFormatter, but it did not work.
What am I doing wrong ?
(The Dispatcher class will be used to store messages and also will allow users to pull messages from it. so I want to backup the data in case of an error and then be able to reload it).
public class Dispatcher
{
private Dictionary<int, Dictionary<int, Dictionary<int, Message>>> m_DataBase;
private Dispatcher()
{
m_DataBase = new Dictionary<int, Dictionary<int, Dictionary<int, Message>>>();
}
public static Dispatcher LoadFromFile()
{
Dispatcher loadedDataBase;
try
{
using (Stream stream = new FileStream(#".\DispatcherDataBase.xml", FileMode.Open))
{
XmlSerializer serializer = new XmlSerializer(typeof(Dispatcher));
loadedDataBase = serializer.Deserialize(stream) as Dispatcher;
}
}
catch (Exception)
{
loadedDataBase = new Dispatcher();
}
return loadedDataBase;
}
public void SaveToFile()
{
FileMode wantedFileModeForStream;
try
{
if (File.Exists(#".\DispatcherDataBase.xml"))
{
wantedFileModeForStream = FileMode.Truncate;
}
else
{
wantedFileModeForStream = FileMode.CreateNew;
}
using (Stream stream = new FileStream(#".\DispatcherDataBase.xml", wantedFileModeForStream))
{
XmlSerializer serializer = new XmlSerializer(this.GetType());
serializer.Serialize(stream, this);
}
}
catch (Exception)
{
}
}
}

You are trying to use XmlSerializer to serialize your Dispatcher type and are encountering three separate problems:
XmlSerializer requires your type to have a public parameterless constructor.
XmlSerializer does not support dictionaries.
XmlSerializer will not serialize non-public members such as m_DataBase.
DataContractSerializer (as well as DataContractJsonSerializer) do not have these limitations. If you mark your Dispatcher type with data contract attributes you will be able to serialize it to XML using this serializer.
Thus if you modify your type as follows:
public static class Constants
{
public const string DataContractNamespace = ""; // Or whatever
}
[DataContract(Name = "Dispatcher", Namespace = Constants.DataContractNamespace)]
public partial class Dispatcher
{
[DataMember]
private Dictionary<int, Dictionary<int, Dictionary<int, Message>>> m_DataBase;
private Dispatcher()
{
m_DataBase = new Dictionary<int, Dictionary<int, Dictionary<int, Message>>>();
}
[System.Runtime.Serialization.OnDeserialized]
void OnDeserializedMethod(System.Runtime.Serialization.StreamingContext context)
{
// Ensure m_DataBase is not null after deserialization (DataContractSerializer does not call the constructor).
if (m_DataBase == null)
m_DataBase = new Dictionary<int, Dictionary<int, Dictionary<int, Message>>>();
}
internal const string FileName = #"DispatcherDataBase.xml";
public static Dispatcher LoadFromFile()
{
Dispatcher loadedDataBase;
try
{
using (var stream = new FileStream(FileName, FileMode.Open))
{
var serializer = new DataContractSerializer(typeof(Dispatcher));
loadedDataBase = serializer.ReadObject(stream) as Dispatcher;
}
}
catch (Exception)
{
loadedDataBase = new Dispatcher();
}
return loadedDataBase;
}
public void SaveToFile()
{
using (var stream = new FileStream(FileName, FileMode.Create))
using (var writer = XmlWriter.Create(stream, new XmlWriterSettings { Indent = true })) // Optional indentation for readability only.
{
var serializer = new DataContractSerializer(this.GetType());
serializer.WriteObject(writer, this);
}
}
}
// Not shown in question, added as an example
[DataContract(Name = "Message", Namespace = Constants.DataContractNamespace)]
public class Message
{
[DataMember]
public string Value { get; set; }
}
You will be able to round-trip your Dispatcher class to XML. Demo fiddle #1 here.
Alternatively, you could use DataContractJsonSerializer and serialize to JSON by simply swapping serializers and eliminating the optional XmlWriter:
public static Dispatcher LoadFromFile()
{
Dispatcher loadedDataBase;
try
{
using (var stream = new FileStream(FileName, FileMode.Open))
{
var serializer = new System.Runtime.Serialization.Json.DataContractJsonSerializer(typeof(Dispatcher));
loadedDataBase = serializer.ReadObject(stream) as Dispatcher;
}
}
catch (Exception)
{
loadedDataBase = new Dispatcher();
}
return loadedDataBase;
}
public void SaveToFile()
{
using (var stream = new FileStream(FileName, FileMode.Create))
{
var serializer = new System.Runtime.Serialization.Json.DataContractJsonSerializer(this.GetType());
serializer.WriteObject(stream, this);
}
}
Demo fiddle #2 here, which results in a fairly simple, clean-looking serialization format:
{"m_DataBase":[{"Key":1,"Value":[{"Key":2,"Value":[{"Key":3,"Value":{"Value":"hello"}}]}]}]}
json.net could also be used to serialize this type if you are willing to use a 3rd party component.

How does marking classes as [Serializable] work in C#?

I made a class called MyData and marked it with [Serializable], but it doesn't seem to work...
Here's MyData.cs:
namespace Forms_Bot_Rebuild
{
[Serializable]
public class MyData
{
private String _Email;
private String _Password;
private String _WorldId;
public MyData(string email, string password, string worldid)
{
_Email = email;
_Password = password;
_WorldId = worldid;
}
public string Email
{ get { return _Email; } set { _Email = value; } }
public string Password
{ get { return _Password; } set { _Password = value; } }
public string WorldId
{ get { return _WorldId; } set { _WorldId = value; } }
}
}
Here's Form1.cs:
private void button1_Click(object sender, EventArgs e)
{
MyData data = new MyData(boxEmail.Text, boxPass.Text, boxWorldId.Text);
string objectNode = Serialize.ToJsonString(data);
File.WriteAllText(Application.StartupPath + #"\LoginData.json", objectNode);
}
private void button2_Click(object sender, EventArgs e)
{
object myData;
myData = Serialize.Deserialize<MyData>(new FileStream(Application.StartupPath + #"\LoginData.json", FileMode.Open));
boxEmail.Text = data.Email;
boxPass.Text = data.Password;
boxWorldId.Text = data.WorldId;
}
Here's Serialize.cs:
public class Serialize
{
public static StreamReader ToStreamReader(object o)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer ser = new DataContractJsonSerializer(o.GetType());
ser.WriteObject(stream, o);
stream.Position = 0;
StreamReader sr = new StreamReader(stream);
return sr;
}
public static string ToJsonString(object o)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer ser = new DataContractJsonSerializer(o.GetType());
ser.WriteObject(stream, o);
stream.Position = 0;
StreamReader sr = new StreamReader(stream);
return sr.ReadToEnd();
}
public static object Deserialize<T>(Stream stream)
{
DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(T));
stream.Position = 0;
T t = (T)ser.ReadObject(stream);
return t;
}
}
Also, this code was given to me by a friend, so I asked him. He couldn't understand it too. I asked another friend, who, in my opinion, knew decent C#, but he couldn't fix it too.
An unhandled exception of type 'System.Runtime.Serialization.SerializationException' occurred in System.Runtime.Serialization.dll
Additional information: Expecting element 'root' from namespace ''.. Encountered 'None' with name '', namespace ''.
Any opinions?

You should use
[DataContract]
instead of
[Serializable]
and mark all the properties with
[DataMember]
(but funnily, on my PC works even with Serializable, only it serializes fields instead of properties... Mmmh I see someone else noticed it: http://pietschsoft.com/post/2008/02/27/NET-35-JSON-Serialization-using-the-DataContractJsonSerializer)
Note that Deserialize could have this signature:
public T Deserialize<T>(Stream stream)
and you should close streams after using them:
MyData myData;
using (var fs = new FileStream(Application.StartupPath + #"\LoginData.json""LoginData.json", FileMode.Open))
{
myData = (MyData)Serialize.Deserialize<MyData>(fs);
}
If instead you really want to know what you wrote in the title:
Each serialization class/method look for "its" serialization attributes (at this time there are at least three families of attributes in Microsoft .NET: [Serializable], the [Xml*] family, the [DataContract]. Plus if you use JSON.NET there are its [Json*] attributes. Stop. So depending on the class/method you want to use to serialize your data, you have to know which attributes it will look for.

DataContractSerializer ignore by default

Want seralization to be an "opt-in" process so that only fields with a [DataMember] attribute show up in the xml.
Looked at DataContractSerializerSettings, but doesn't seem to have anything. Tried below for kicks and giggles, but the xml still contained properties that did not have the [DataMember] attribute...
DataContractSerializer writer =
new DataContractSerializer( typeof( ProductAAnimals ),
new DataContractSerializerSettings()
{
IgnoreExtensionDataObject = true
} );

With .NET 3.5 SP1, Members decorated with [DataMember], any public read/write property will be serialized automatically by the DataContractSerializer. In order to opt-out, you have to go with private members. This wasn't the case before .NET 3.5 where only members decorated with [DataMemeber] are serialized.
More on what's new in .NET 3.5 SP1
Blog.

O
that's not true try to take a look at this code from msdn and try to uncomment and comment the datamember attribute in class person for the property LastName for example an yot will see that's not serialized as #Nair saied . I think that you are trying to write to the same file but the file it's locked so you will think that all properties are serialized. try to change the name every time and you will see that property not marked with datamember attribute will not be serialized
using System;
using System.Collections;
using System.IO;
using System.Runtime.Serialization;
using System.Xml;
// You must apply a DataContractAttribute or SerializableAttribute
// to a class to have it serialized by the DataContractSerializer.
[DataContract()]
class Person : IExtensibleDataObject
{
private string LastNameValue;
// Apply the DataMemberAttribute to fields (or properties)
// that must be serialized.
[DataMember()]
public string FirstName;
//[DataMember()]
public string LastName
{
get { return LastNameValue; }
set { LastNameValue = value; }
}
[DataMember(Name = "ID")]
public int IdNumber;
// Note that you can apply the DataMemberAttribute to
// a private field as well.
[DataMember]
private string Secret;
public Person(string newfName, string newLName, int newIdNumber)
{
FirstName = newfName;
LastName = newLName;
IdNumber = newIdNumber;
Secret = newfName + newLName + newIdNumber;
}
// The extensionDataValue field holds data from future versions
// of the type. This enables this type to be compatible with
// future versions. The field is required to implement the
// IExtensibleDataObject interface.
private ExtensionDataObject extensionDatavalue;
public ExtensionDataObject ExtensionData
{
get
{
return extensionDatavalue;
}
set
{
extensionDatavalue = value;
}
}
}
public class Test
{
public static void Main(string[] args)
{
try
{
WriteObject(#"DataMemberAttributeExample.xml");
ReadObject(#"DataMemberAttributeExample.xml");
}
catch (Exception exc)
{
Console.WriteLine(
"The serialization operation failed: {0} StackTrace: {1}",
exc.Message, exc.StackTrace);
}
finally
{
Console.WriteLine("Press <Enter> to exit....");
Console.ReadLine();
}
}
public static void WriteObject(string filename)
{
// Create a new instance of the Person class.
Person p1 = new Person("Zighetti", "Barbara", 101);
FileStream writer = new FileStream(filename,
FileMode.OpenOrCreate);
DataContractSerializer ser =
new DataContractSerializer(typeof(Person));
ser.WriteObject(writer, p1);
writer.Close();
}
public static void ReadObject(string filename)
{
// Deserialize an instance of the Person class
// from an XML file.
FileStream fs = new FileStream(filename,
FileMode.OpenOrCreate);
DataContractSerializer ser =
new DataContractSerializer(typeof(Person));
// Deserialize the data and read it from the instance.
Person deserializedPerson = (Person)ser.ReadObject(fs);
fs.Close();
Console.WriteLine(String.Format("{0} {1}, ID: {2}",
deserializedPerson.FirstName, deserializedPerson.LastName,
deserializedPerson.IdNumber));
}
}

Deserialisation doesn't rebuild dictionaries that are members of classes

I've got some serialisation code set up as follows:
static void SerialiseObject(Object o, String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Create);
formatter.Serialize(stream, o);
stream.Close();
}
static Object DeserialiseObject(String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Open, FileAccess.Read);
Object o = (Object)formatter.Deserialize(stream);
stream.Close();
return o;
}
And a class with the following member defined:
[Serializable]
public class CircuitModel
{
public Dictionary<String, Bus> Buses { protected set; get; }
...
}
I populate the Dictionary, and then the following code successfully serialises and deserialises the dictionary, with all Bus objects intact:
SerialiseObject(CircuitModel.Buses, "temp.bin");
Object o = DeserialiseObject("temp.bin");
But when I try to do the same for CircuitModel:
SerialiseObject(CircuitModel, "temp.bin");
Object o = DeserialiseObject("temp.bin");
CircuitModel.Buses has been initialised, but is empty.
I've also tried implementing serialisation with ISerializable (for the Bus and CircuitModel classes) and had exactly the same problem
Any idea as to why this would be happening?

I think you have something more sinister going on with your child collection because binary serialization of Dictionaries within classes does work just fine.
[TestFixture]
public class SerializeTest
{
[Test]
public void TestSer()
{
var parent = new Parent
{
Name = "Test"
};
parent.Children.Add("Child1", new Child {Name = "Child1"});
parent.Children.Add( "Child2", new Child { Name = "Child2" } );
SerialiseObject(parent, "test.bin");
var copy = DeserialiseObject("test.bin") as Parent;
Assert.IsNotNull(copy);
Assert.AreEqual(2, copy.Children.Count);
Assert.IsTrue(copy.Children.ContainsKey("Child1"));
Assert.AreEqual("Child1", copy.Children["Child1"].Name);
}
static void SerialiseObject( Object o, String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Create );
formatter.Serialize( stream, o );
stream.Close();
}
static Object DeserialiseObject( String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Open, FileAccess.Read );
Object o = (Object) formatter.Deserialize( stream );
stream.Close();
return o;
}
[Serializable]
private class Parent
{
public string Name { get; set; }
public Dictionary<string, Child> Children { get; protected set; }
public Parent()
{
Children = new Dictionary<string, Child>();
}
}
[Serializable]
private class Child
{
public string Name { get; set; }
}
}
The children deserialize with the parent and contain the details they were initialized with. I would check any code that is setting your Buses collection. My example just did it in the constructor of the parent class, but it may be possible that you have rogue code setting it after it's been deserialized?

Dictionaries are not serializable. Remove the dictionary if you need to serialize that data, and replace it by a list of a custom class that contains the data in the dictionary:
[Serializable]
public class BusItem
{
public string Name {get;set;}
public Bus Bus {get;set;}
}
Edit: I just found out you can actually serialize Dictionaries using the DataContractSerializer instead.
http://theburningmonk.com/2010/05/net-tips-xml-serialize-or-deserialize-dictionary-in-csharp/

If you are talking about XML serialization, it might be because Dictionary is not serializable to XML. Look at Why isn't there an XML-serializable dictionary in .NET.

Querying serialized object file

Is there anyway to achieve that without loading the whole file into memory? If so, what do you suggest me to do?
Class implementation:
[Serializable()]
public class Car
{
public string Brand { get; set; }
public string Model { get; set; }
}
[Serializable()]
public class CarCollection : List<Car>
{
}
Serialization to file:
CarCollection cars = new CarCollection
{
new Cars{ Brand = "BMW", Model = "7.20" },
new Cars{ Brand = "Mercedes", Model = "CLK" }
};
using (Stream stream = File.Open("data", FileMode.Create))
{
BinaryFormatter bin = new BinaryFormatter();
bin.Serialize(stream, cars);
}

If you serialize to XML you can use a SAX parser (XmlReader class), which will read from a stream seqentially.

To deserialize the collection one object at a time, you also need to serialize it one at a time.
Simplest way is to define your own generic class:
public static class StreamSerializer
{
public static void Serialize<T>(IList<T> list, string filename)
{
using (Stream stream = File.Open(filename, FileMode.Create))
{
BinaryFormatter bin = new BinaryFormatter();
// seralize each object separately
foreach (var item in list)
bin.Serialize(stream, item);
}
}
public static IEnumerable<T> Deserialize<T>(string filename)
{
using (Stream stream = File.Open(filename, FileMode.Open))
{
BinaryFormatter bin = new BinaryFormatter();
// deserialize each object separately, and
// return them one at a time
while (stream.Position < stream.Length)
yield return (T)bin.Deserialize(stream);
}
}
}
Then you can simply write:
CarsCollection cars = new CarsCollection
{
new Cars{ Brand = "BMW", Model = "7.20" },
new Cars{ Brand = "Mercedes", Model = "CLK" }
};
// note that you cannot serialize the entire list if
// you want to query without loading - it must be symmetrical
StreamSerializer.Serialize(cars, "data.bin");
// the following expression iterates through objects, processing one
// at a time. "First" method is a good example because it
// breaks early.
var bmw = StreamSerializer
.Deserialize<Cars>("data.bin")
.First(c => c.Brand == "BMW");
A slightly more complex case might be if your CarsCollection belongs to a different class. In that case, you will need to implement ISerializable, but the principle is similar.
On a side note, usual convention is not to name entities in plural (i.e. Cars should be named Car).

Generally you can use some sort of reader (StreamReader, BinaryReader, ...) together with BufferedStream.