Is there anyway to achieve that without loading the whole file into memory? If so, what do you suggest me to do?
Class implementation:
[Serializable()]
public class Car
{
public string Brand { get; set; }
public string Model { get; set; }
}
[Serializable()]
public class CarCollection : List<Car>
{
}
Serialization to file:
CarCollection cars = new CarCollection
{
new Cars{ Brand = "BMW", Model = "7.20" },
new Cars{ Brand = "Mercedes", Model = "CLK" }
};
using (Stream stream = File.Open("data", FileMode.Create))
{
BinaryFormatter bin = new BinaryFormatter();
bin.Serialize(stream, cars);
}
If you serialize to XML you can use a SAX parser (XmlReader class), which will read from a stream seqentially.
To deserialize the collection one object at a time, you also need to serialize it one at a time.
Simplest way is to define your own generic class:
public static class StreamSerializer
{
public static void Serialize<T>(IList<T> list, string filename)
{
using (Stream stream = File.Open(filename, FileMode.Create))
{
BinaryFormatter bin = new BinaryFormatter();
// seralize each object separately
foreach (var item in list)
bin.Serialize(stream, item);
}
}
public static IEnumerable<T> Deserialize<T>(string filename)
{
using (Stream stream = File.Open(filename, FileMode.Open))
{
BinaryFormatter bin = new BinaryFormatter();
// deserialize each object separately, and
// return them one at a time
while (stream.Position < stream.Length)
yield return (T)bin.Deserialize(stream);
}
}
}
Then you can simply write:
CarsCollection cars = new CarsCollection
{
new Cars{ Brand = "BMW", Model = "7.20" },
new Cars{ Brand = "Mercedes", Model = "CLK" }
};
// note that you cannot serialize the entire list if
// you want to query without loading - it must be symmetrical
StreamSerializer.Serialize(cars, "data.bin");
// the following expression iterates through objects, processing one
// at a time. "First" method is a good example because it
// breaks early.
var bmw = StreamSerializer
.Deserialize<Cars>("data.bin")
.First(c => c.Brand == "BMW");
A slightly more complex case might be if your CarsCollection belongs to a different class. In that case, you will need to implement ISerializable, but the principle is similar.
On a side note, usual convention is not to name entities in plural (i.e. Cars should be named Car).
Generally you can use some sort of reader (StreamReader, BinaryReader, ...) together with BufferedStream.
Related
i have an object of following class
[XmlRoot("http://schemas.abc.com")]
[DataContract]
public class Employee
{
[XmlAttribute]
[DataMember]
public List<string> Positions { get; set; }
}
Positions contains following two strings "Project Manager" and "Senior Project Manager"
I serialized this object through following method and saved in DB
public static string Serialize(Employee entity)
{
var memoryStream = new MemoryStream();
var serializer = new XmlSerializer(typeof(Employee));
using (var xmlTextWriter = new XmlTextWriter(memoryStream,
Encoding.Unicode))
{
serializer.Serialize(xmlTextWriter, entity);
xmlTextWriter.Close();
}
return UnicodeByteArrayToString(memoryStream.ToArray());
}
private static string UnicodeByteArrayToString(byte[] input)
{
var constructedString = Encoding.Unicode.GetString(input);
return constructedString;
}
Then i am retrieving and deserializing the object through following method
public static Employee Deserialize(string str)
{
var data = StringToUnicodeByteArray(str);
var memoryStream = new MemoryStream(data);
var serializer = new XmlSerializer(typeof(Employee));
var xmlTextReader = new XmlTextReader(memoryStream);
return (Employee)serializer.Deserialize(xmlTextReader);
}
private static byte[] StringToUnicodeByteArray(string input)
{
var byteArray = Encoding.Unicode.GetBytes(input);
return byteArray;
}
Now i am getting 5 objects of strings in position list
"Project", "Manager", "Senior", "Project", "Manager"
deserialization is creating new string object after every space
Quick help will be much appreciated
Answer from Eugene Podskal
think that you should remove the XmlAttribute attribute. XmlSerializer doesn't know how to parse values in attribute string other than by splitting it on whitespace. Or you can create a property that will give you String that allows to unambiguously separate individual items in it (like with commas or whatever), and that can parse individual items from the String it is set to. This propery will be serializable, while actual List will be XmlIgnore
I know this is a popular topic and I have researched extensively without finding an answer to my problem.
I have a base class IntroductionAction and 2 derived classes IntroductionActionComplex and IntroductionActionSimple. I have a list of IntroductionAction objects to which I have added objects of both of the derived types. My classes are as follows:
[XmlInclude(typeof(IntroductionActionComplex))]
[XmlInclude(typeof(IntroductionActionSimple))]
public class IntroductionAction
{
public IntroductionAction() { }
}
public class IntroductionActionComplex : IntroductionAction
{
[XmlIgnore]
public string name { get; set; }
[XmlElement(ElementName = "QuestionString")]
public string question { get; set; }
[XmlElement(ElementName = "AnswerString")]
public List<string> answerStrings { get; set; }
public IntroductionActionComplex()
{
name = string.Empty;
question = null;
answerStrings = new List<string>();
}
}
public class IntroductionActionSimple : IntroductionAction
{
[XmlIgnore]
public string name { get; set; }
[XmlText]
public string Value { get; set; }
public IntroductionActionSimple()
{
Value = string.Empty;
}
}
I then create the List as follows
[XmlElement("IntroductionAction")]
public List<IntroductionAction> introductionActions { get; set; }
I am using XmlSerializer and everything serializes correctly. This is the resulting XML of the list containing one of each of the derived classes which is correct.
<IntroductionAction>
<QuestionString>
test
</QuestionString>
<AnswerString>
test
</AnswerString>
<AnswerString>
test
</AnswerString>
</IntroductionAction>
<IntroductionAction>
test
</IntroductionAction>
This XML file is going onto a device which doesn't read it as XML but just searches for the tags and does whatever work it needs to do and because of that the file can't contain any XSI or XSD tags, indentation, etc that is usually associated with proper XML.
My deserialization code is straight forward:
public static T Deserialize_xml_Config<T>(string file1, T obj)
{
XmlSerializer deserializer = new XmlSerializer(obj.GetType());
using (TextReader reader = new StreamReader(file1))
{
return (T)deserializer.Deserialize(reader);
}
}
Finally to my problem. When I deserialize, it is being deserialized to the base class IntroductionAction and not to the derived classes.
These IntroductionAction classes are just part of a much larger object that I am serializing/deserializing. I have tried making the base class abstract since it contains no functionality but I get an error on deserialization saying
The specified type is abstract: name='IntroductionAction'
Despite my XmlIncludes it seems unable to find the derived classes.
I have tried adding the types to the serializer but that didn't work.
Any help is much appreciated.
Edit:
This is what I mean by adding the types to the serializer
XmlSerializer deserializer = new XmlSerializer(obj.GetType(), new Type [] { typeof(IntroductionActionComplex), typeof(IntroductionActionSimple) });
using (TextReader reader = new StreamReader(file1))
{
return (T)deserializer.Deserialize(reader);
}
Also my attempt at using XmlAttributeOverrides:
XmlAttributeOverrides attrOverrides = new XmlAttributeOverrides();
var attrs = new XmlAttributes();
XmlElementAttribute attr = new XmlElementAttribute();
attr.ElementName = "IntroductionAction";
attr.Type = typeof(IntroductionActionComplex);
attrs.XmlElements.Add(attr);
attr.ElementName = "IntroductionAction";
attr.Type = typeof(IntroductionActionSimple);
attrs.XmlElements.Add(attr);
attrOverrides.Add(typeof(IntroductionAction), "IntroductionAction", attrs);
XmlSerializer deserializer = new XmlSerializer(obj.GetType(), attrOverrides);
using (TextReader reader = new StreamReader(file1))
{
return (T)deserializer.Deserialize(reader);
}
I think you are pretty close. Below is the full example of saving and loading the XML file based on derived class types. This will save the nodes as the derived type itself, so loading back in will keep the desired type, rather than convert back to the base type. You'll probably need to add exception handling, this was just a quick solution. I did not change your base IntroductionAction or the derived IntroductionActionComplex / IntroductionActionSimple classes.
public class RootNode
{
[XmlElement("IntroductionAction")]
public List<IntroductionAction> introductionActions { get; set; }
public RootNode()
{
introductionActions = new List<IntroductionAction>();
}
private static XmlAttributeOverrides GetXmlAttributeOverrides()
{
XmlAttributeOverrides xml_attr_overrides = new XmlAttributeOverrides();
XmlAttributes xml_attrs = new XmlAttributes();
xml_attrs.XmlElements.Add(new XmlElementAttribute(typeof(IntroductionActionComplex)));
xml_attrs.XmlElements.Add(new XmlElementAttribute(typeof(IntroductionActionSimple)));
xml_attr_overrides.Add(typeof(RootNode), "introductionActions", xml_attrs);
return xml_attr_overrides;
}
// Add exception handling
public static void SaveToFile(RootNode rootNode, string fileName)
{
using (MemoryStream mem_stream = new MemoryStream())
{
XmlSerializer serializer = new XmlSerializer(rootNode.GetType(), RootNode.GetXmlAttributeOverrides());
serializer.Serialize(mem_stream, rootNode);
using (BinaryWriter output = new BinaryWriter(new FileStream(fileName, FileMode.Create)))
{
output.Write(mem_stream.ToArray());
}
}
}
// Add exception handling
public static RootNode LoadFromFile(string fileName)
{
if (File.Exists(fileName))
{
using (FileStream file = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (TextReader reader = new StreamReader(file))
{
XmlSerializer serializer = new XmlSerializer(typeof(RootNode), RootNode.GetXmlAttributeOverrides());
return (RootNode)serializer.Deserialize(reader);
}
}
}
return null;
}
}
Test program:
internal class Program
{
private static void Main(string[] args)
{
RootNode obj = new RootNode();
obj.introductionActions.Add(new IntroductionActionComplex() { question = "qTest", answerStrings = { "aTest1", "aTest2" }, name = "aName1" });
obj.introductionActions.Add(new IntroductionActionSimple() { name = "aName2", Value = "aValue" });
RootNode.SaveToFile(obj, "Test.xml");
RootNode obj2 = RootNode.LoadFromFile("Test.xml");
}
}
I have a SortedList, the key is Contact.name and the value, is an object, Contact. I´m writing this list to a file, with BinaryWriter and I don´t have any trouble but now I want to read this list, and after, look for an specific contact. I don´t know how to do it. I think that I must read the file and after that look for the contact, but how can I fill the SortedList with the binary code saved in the file?
Here's a quick example on how to do, both, read and write from a binary file.
static void Main(string[] args)
{
WriteContacts(new List<Contact>( new []{ new Contact { ID = 1, Name = "Juan", Age = 34 }, new Contact { Name = "Pedro", Age = 23, ID = 2 } }));
FindContactInFile("Juan");
FindContactInFile("Mario");
Console.ReadKey();
}
private static void FindContactInFile(string name)
{
IFormatter formatter = new BinaryFormatter();
using (Stream s = new FileStream("contacts.bin", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var contacts = (List<Contact>)formatter.Deserialize(s);
var person = contacts.Where(x=>x.Name==name).FirstOrDefault();
if (person != null)
Console.WriteLine("Persona encontrada: {0}", person.Name);
else
Console.WriteLine("{0} no fue encontrado en el archivo.", name);
}
}
private static void WriteContacts(List<Contact> contacts)
{
IFormatter formatter = new BinaryFormatter();
using (Stream s = new FileStream("contacts.bin", FileMode.Create, FileAccess.Write, FileShare.None))
{
formatter.Serialize(s, contacts);
}
}
}
[Serializable]
class Contact
{
public int ID { get; set; }
public string Name { get; set; }
public int Age { get; set; }
}
Many things can be improved there, like not reading the entire file every time you search for someone. Or not reading the entire file at once to begin with... In any case, the key concept here is that in order to store something in a binary file you need to serialize the object. For this you can use one of the BinaryFormatters provided by .NET (as I did above) and to read back from the file, you simply do the opposite.
Sounds like you want BinaryReader which will take the File output from BinaryWriter and deserialize into your original type.
This is a good example of using both on msdn
I've got some serialisation code set up as follows:
static void SerialiseObject(Object o, String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Create);
formatter.Serialize(stream, o);
stream.Close();
}
static Object DeserialiseObject(String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Open, FileAccess.Read);
Object o = (Object)formatter.Deserialize(stream);
stream.Close();
return o;
}
And a class with the following member defined:
[Serializable]
public class CircuitModel
{
public Dictionary<String, Bus> Buses { protected set; get; }
...
}
I populate the Dictionary, and then the following code successfully serialises and deserialises the dictionary, with all Bus objects intact:
SerialiseObject(CircuitModel.Buses, "temp.bin");
Object o = DeserialiseObject("temp.bin");
But when I try to do the same for CircuitModel:
SerialiseObject(CircuitModel, "temp.bin");
Object o = DeserialiseObject("temp.bin");
CircuitModel.Buses has been initialised, but is empty.
I've also tried implementing serialisation with ISerializable (for the Bus and CircuitModel classes) and had exactly the same problem
Any idea as to why this would be happening?
I think you have something more sinister going on with your child collection because binary serialization of Dictionaries within classes does work just fine.
[TestFixture]
public class SerializeTest
{
[Test]
public void TestSer()
{
var parent = new Parent
{
Name = "Test"
};
parent.Children.Add("Child1", new Child {Name = "Child1"});
parent.Children.Add( "Child2", new Child { Name = "Child2" } );
SerialiseObject(parent, "test.bin");
var copy = DeserialiseObject("test.bin") as Parent;
Assert.IsNotNull(copy);
Assert.AreEqual(2, copy.Children.Count);
Assert.IsTrue(copy.Children.ContainsKey("Child1"));
Assert.AreEqual("Child1", copy.Children["Child1"].Name);
}
static void SerialiseObject( Object o, String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Create );
formatter.Serialize( stream, o );
stream.Close();
}
static Object DeserialiseObject( String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Open, FileAccess.Read );
Object o = (Object) formatter.Deserialize( stream );
stream.Close();
return o;
}
[Serializable]
private class Parent
{
public string Name { get; set; }
public Dictionary<string, Child> Children { get; protected set; }
public Parent()
{
Children = new Dictionary<string, Child>();
}
}
[Serializable]
private class Child
{
public string Name { get; set; }
}
}
The children deserialize with the parent and contain the details they were initialized with. I would check any code that is setting your Buses collection. My example just did it in the constructor of the parent class, but it may be possible that you have rogue code setting it after it's been deserialized?
Dictionaries are not serializable. Remove the dictionary if you need to serialize that data, and replace it by a list of a custom class that contains the data in the dictionary:
[Serializable]
public class BusItem
{
public string Name {get;set;}
public Bus Bus {get;set;}
}
Edit: I just found out you can actually serialize Dictionaries using the DataContractSerializer instead.
http://theburningmonk.com/2010/05/net-tips-xml-serialize-or-deserialize-dictionary-in-csharp/
If you are talking about XML serialization, it might be because Dictionary is not serializable to XML. Look at Why isn't there an XML-serializable dictionary in .NET.
When writing to a stream the DataContractSerializer uses an encoding different from Unicode-16. If I could force it to write/read Unicode-16 I could store it in a SQL CE's binary column and read it with SELECT CONVERT(nchar(1000), columnName). But the way it is, I can't read it, except programatically.
Can I change the encoding used by System.Runtime.Serialization.DataContractSerializer?
The DataContractSerializer's WriteObject method has overloads which write to a Stream or to a XmlWriter (and XmlDictionaryWriter). The Stream overload will default to UTF-8, so you'll need to use another one. Using a XML Writer instance which writes the XML in UTF-16 do what you needs, so you can either do what #Phil suggested, or you can use the writer returned by XmlDictionaryWriter.CreateTextWriter for which you pass an Encoding.Unicode as a parameter.
public class StackOverflow_10089682
{
[DataContract(Name = "Person", Namespace = "http://my.namespace")]
public class Person
{
[DataMember]
public string Name { get; set; }
[DataMember]
public int Age { get; set; }
}
public static void Test()
{
MemoryStream ms = new MemoryStream();
XmlDictionaryWriter writer = XmlDictionaryWriter.CreateTextWriter(ms, Encoding.Unicode);
DataContractSerializer dcs = new DataContractSerializer(typeof(Person));
Person instance = new Person { Name = "John Doe", Age = 33 };
dcs.WriteObject(writer, instance);
writer.Flush(); // Don't forget to Flush the writer here
Console.WriteLine("Decoding using UTF-16: {0}", Encoding.Unicode.GetString(ms.ToArray()));
}
}
Have you tried using XmlWriterSettings? Something like
var s = new DataContractSerializer (typeof(Thing));
using(var wr = XmlTextWriter.Create(
#"test.xml", new XmlWriterSettings{Encoding=Encoding.UTF32}))
{
s.WriteObject(wr, new Thing{Foo="bar"});
}
public class Thing
{
public string Foo { get; set; }
}
Specify the Encoding you require.