I got 2 user-defined classes, called Event and Image, Event has a property stored a list of EventImage, called EventImages. And in Image class, there's a byte[] type property which store the byte[] of one Image file.
Here's the definitions of the 2 classes :
[Serializable]
public Class Event
{
public String Name {get; set;}
public DateTime EventTime { get; set;}
public List<Image> EventImages { get; set; }
...
}
[Serializable]
public Class Image
{
public DateTime ImageTime { get; set;}
public byte[] pData {get; set;}
...
}
Now my question is, I want to serialize my Event object to byte[], and I expect that the whole content of it will be serialize, too, but it seems that I failed.
Here's my code to do Serialization :
public static byte[] ObjectToByteArray(object obj)
{
if (obj == null)
{
return null;
}
else
{
BinaryFormatter bF = new BinaryFormatter();
using (MemoryStream mS = new MemoryStream())
{
bF.Serialize(mS, obj);
return mS.ToArray();
}
}
}
and here's the code for verification :
Console.WriteLine(ObjectToByteArray(Event));
Console.WriteLine(ObjectToByteArray(Event.EventImage));
Console.WriteLine(ObjectToByteArray(Event.EventImages.FirstOrDefault().pData));
and the results are(just assumption value) :
100 200 300
But I expect the result should be 600(100+200+300), 500(200+300) and 300.
So, I think my serialization doesn't really serialize the whole content, it just serialize properties with Basic Types, but without the nested objects, am I right?
I've searched lots of posts, and I found plenty of answers to similar questions mentioned "XML Serialization", but I'm not sure whether its helpful or not. Need I use that or is there any other better way? Thanks in advance!
Quickly made a test to check equality. It passes.
Serializes and deserializes correctly.
Conclusion, don't judge whether something is happening or not until you've tested it.
public static void Run()
{
var i = new Image
{
ImageTime = DateTime.UtcNow,
pData = Guid.NewGuid().ToByteArray()
};
var e = new Event
{
Name = Guid.NewGuid().ToString(),
EventTime = DateTime.UtcNow,
EventImages = new List<Image> {i}
};
var bytes = ObjectToByteArray(e);
var e2 = ObjectFromByteArray(bytes);
Console.WriteLine(e.Equals(e2));
}
public static byte[] ObjectToByteArray(object obj)
{
if (obj == null)
{
return null;
}
var bF = new BinaryFormatter();
using (var mS = new MemoryStream())
{
bF.Serialize(mS, obj);
return mS.ToArray();
}
}
public static object ObjectFromByteArray(byte[] bytes)
{
if (bytes == null)
{
return null;
}
var bF = new BinaryFormatter();
using (var mS = new MemoryStream(bytes))
{
return bF.Deserialize(mS);
}
}
Related
I have a very complex csla object. This object has been serialized to Database in binary. One of the child objects changes and I cannot deserialize the object anymore. (When I try to deserialize it, the properties contain junk or are not initialized because the deserialization did'nt work in the new class.) I'll post a simplified example here.
[serializable]
public class Head : BusinessBase<Head>
{
private static PropertyInfo<int> _eyesCountProperty = RegisterProperty<int>(new PropertyInfo<int>("EyesCount"));
public int EyesCount
{
get { return GetProperty(_eyesCountProperty ); }
set { SetProperty(_eyesCountProperty , value); }
}
}
[serializable]
public class Person : BusinessBase<Person>
{
private static PropertyInfo<string> _firstNameProperty = RegisterProperty<string>(new PropertyInfo<string>("FirstName"));
public string FirstName
{
get { return GetProperty(_firstNameProperty ); }
set { SetProperty(_firstNameProperty , value); }
}
}
public Head MyHead { get; set; }
}
So let's say I have an instance of "Person" class and I serialize it to the database.
Here's our serializing and deserializing methods.
public static byte[] ConvertToByteArray(object theObject)
{
byte[] result = null;
BinaryFormatter bf = new BinaryFormatter();
using(MemoryStream ms = new MemoryStream())
{
bf.Serialize(ms, theObject);
result = ms.ToArray();
}
return result;
}
public static object ConvertFromByteArray(byte[] serializedObject)
{
object result = null;
using(MemoryStream ms = new MemoryStream(serializedObject))
{
BinaryFormatter bf = new BinaryFormatter();
result = bf.Deserialize(ms);
}
return result;
}
Now let's say that the Head class changes and I now have the noseCount and mouthCount in it. So I try to rename it as "HeadV1" and create a "HeadV2" class with the new properties in it. (I would need to do a "PersonV1" class with the "HeadV1" property and a "PersonV2" class with the "HeadV2".)
In the PersonV2 class, I would like to have an "OnDeserializing" method that would let me know if the item I'm trying to deserialize is of type "PersonV1" or "PersonV2" in order to deserialize it the right way.
[OnDeserializing()]
internal void OnDeserializingMethod(StreamingContext context)
{
// Determine if data is of type "PersonV1" or "PersonV2"
}
But I'm stuck, I don't know how to do it and I can't seem to find how to do it. Is there any way to do so? I don't seem to have access to the data inside the "OnDeserializing" method?
Is there possible to writte protobuf serialized content directly to SharpZipLib stream? When I try to do this, looks like the provided stream is not filled with the data from protobuf. Later I would need to get back the deserialized entity from provided Zip stream.
My code looks like this:
private byte[] ZipContent(T2 content)
{
const short COMPRESSION_LEVEL = 4; // 0-9
const string ENTRY_NAME = "DefaultEntryName";
byte[] result = null;
if (content == null)
return result;
IStreamSerializerProto<T2> serializer = this.GetSerializer(content.GetType());
using (MemoryStream outputStream = new MemoryStream())
{
using (ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream))
{
zipOutputStream.SetLevel(COMPRESSION_LEVEL);
ZipEntry entry = new ZipEntry(ENTRY_NAME);
entry.DateTime = DateTime.Now;
zipOutputStream.PutNextEntry(entry);
serializer.Serialize(zipOutputStream, content);
}
result = outputStream.ToArray();
}
return result;
}
private class ProtobufStreamSerializer<T3> : IStreamSerializerProto<T3>
{
public ProtobufStreamSerializer()
{
ProtoBuf.Serializer.PrepareSerializer<T3>();
}
public void Serialize(Stream outputStream, T3 content)
{
Serializer.Serialize(outputStream, content);
}
public T3 Deserialize(Stream inputStream)
{
T3 deserializedObj;
using (inputStream)
{
deserializedObj = ProtoBuf.Serializer.Deserialize<T3>(inputStream);
}
return deserializedObj;
}
}
Sample of a class which I'm trying to serialize:
[Serializable]
[ProtoContract]
public class Model
{
[XmlElement("ModelCode")]
[ProtoMember(1)]
public int ModelCode { get; set; }
...
}
This is the problem, I believe (in the original code in the question):
public void Serialize(Stream outputStream, T3 content)
{
using (var stream = new MemoryStream())
{
Serializer.Serialize(stream, content);
}
}
You're completely ignoring outputStream, instead writing the data just to a new MemoryStream which is then ignored.
I suspect you just want:
public void Serialize(Stream outputStream, T3 content)
{
Serializer.Serialize(outputStream, content);
}
I'd also suggest removing the using statement from your Deserialize method: I'd expect the caller to be responsible for disposing of the input stream when they're finished with it. Your method can be simplified to:
public T3 Deserialize(Stream inputStream)
{
return ProtoBuf.Serializer.Deserialize<T3>(inputStream);
}
The code (with the edit pointed out by Jon) looks fine. Here it is working:
static void Main()
{
var obj = new Bar{ X = 123, Y = "abc" };
var wrapper = new Foo<Bar>();
var blob = wrapper.ZipContent(obj);
var clone = wrapper.UnzipContent(blob);
}
where Bar is:
[ProtoContract]
class Bar
{
[ProtoMember(1)]
public int X { get; set; }
[ProtoMember(2)]
public string Y { get; set; }
}
and Foo<T> is your class (I didn't know the name), where I have added:
public T2 UnzipContent(byte[] data)
{
using(var ms = new MemoryStream(data))
using(var zip = new ZipInputStream(ms))
{
var entry = zip.GetNextEntry();
var serializer = this.GetSerializer(typeof(T2));
return serializer.Deserialize(zip);
}
}
Also, note that compression is double-edged. In the example I give above, the underlying size (i.e. if we just write to a MemoryStream) is 7 bytes. ZipInputStream "compresses" this 7 bytes down to 179 bytes. Compression works best on larger objects, usually when there is lots of text content.
I've got some serialisation code set up as follows:
static void SerialiseObject(Object o, String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Create);
formatter.Serialize(stream, o);
stream.Close();
}
static Object DeserialiseObject(String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Open, FileAccess.Read);
Object o = (Object)formatter.Deserialize(stream);
stream.Close();
return o;
}
And a class with the following member defined:
[Serializable]
public class CircuitModel
{
public Dictionary<String, Bus> Buses { protected set; get; }
...
}
I populate the Dictionary, and then the following code successfully serialises and deserialises the dictionary, with all Bus objects intact:
SerialiseObject(CircuitModel.Buses, "temp.bin");
Object o = DeserialiseObject("temp.bin");
But when I try to do the same for CircuitModel:
SerialiseObject(CircuitModel, "temp.bin");
Object o = DeserialiseObject("temp.bin");
CircuitModel.Buses has been initialised, but is empty.
I've also tried implementing serialisation with ISerializable (for the Bus and CircuitModel classes) and had exactly the same problem
Any idea as to why this would be happening?
I think you have something more sinister going on with your child collection because binary serialization of Dictionaries within classes does work just fine.
[TestFixture]
public class SerializeTest
{
[Test]
public void TestSer()
{
var parent = new Parent
{
Name = "Test"
};
parent.Children.Add("Child1", new Child {Name = "Child1"});
parent.Children.Add( "Child2", new Child { Name = "Child2" } );
SerialiseObject(parent, "test.bin");
var copy = DeserialiseObject("test.bin") as Parent;
Assert.IsNotNull(copy);
Assert.AreEqual(2, copy.Children.Count);
Assert.IsTrue(copy.Children.ContainsKey("Child1"));
Assert.AreEqual("Child1", copy.Children["Child1"].Name);
}
static void SerialiseObject( Object o, String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Create );
formatter.Serialize( stream, o );
stream.Close();
}
static Object DeserialiseObject( String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Open, FileAccess.Read );
Object o = (Object) formatter.Deserialize( stream );
stream.Close();
return o;
}
[Serializable]
private class Parent
{
public string Name { get; set; }
public Dictionary<string, Child> Children { get; protected set; }
public Parent()
{
Children = new Dictionary<string, Child>();
}
}
[Serializable]
private class Child
{
public string Name { get; set; }
}
}
The children deserialize with the parent and contain the details they were initialized with. I would check any code that is setting your Buses collection. My example just did it in the constructor of the parent class, but it may be possible that you have rogue code setting it after it's been deserialized?
Dictionaries are not serializable. Remove the dictionary if you need to serialize that data, and replace it by a list of a custom class that contains the data in the dictionary:
[Serializable]
public class BusItem
{
public string Name {get;set;}
public Bus Bus {get;set;}
}
Edit: I just found out you can actually serialize Dictionaries using the DataContractSerializer instead.
http://theburningmonk.com/2010/05/net-tips-xml-serialize-or-deserialize-dictionary-in-csharp/
If you are talking about XML serialization, it might be because Dictionary is not serializable to XML. Look at Why isn't there an XML-serializable dictionary in .NET.
I have been using some code to create MTOM by using code from MSDN.
It seems that there is an error and I cannot understand where the problem lies as one of the users on the forum pointed out that there is an error.
The file (JPEG) data get corrupted after a de-serialization. The complete code is listed below.
public class Post_7cb0ff86_5fe1_4266_afac_bcb91eaca5ec
{
[DataContract()]
public partial class TestAttachment
{
private byte[] fileField;
private string filenameField;
[DataMember()]
public byte[] File
{
get
{
return this.fileField;
}
set
{
this.fileField = value;
}
}
[DataMember()]
public string Filename
{
get
{
return this.filenameField;
}
set
{
this.filenameField = value;
}
}
}
public static void Test()
{
string Filename = "Image.jpg";
byte[] file = File.ReadAllBytes(Filename);
TestAttachment Attachment = new TestAttachment();
Attachment.Filename = Filename;
Attachment.File = file;
MemoryStream MTOMInMemory = new MemoryStream();
XmlDictionaryWriter TW = XmlDictionaryWriter.CreateMtomWriter(MTOMInMemory, Encoding.UTF8, Int32.MaxValue, "");
DataContractSerializer DCS = new DataContractSerializer(Attachment.GetType());
DCS.WriteObject(TW, Attachment);
TW.Flush();
Console.WriteLine(Encoding.UTF8.GetString(MTOMInMemory.ToArray()));
var v = DeserializeMTOMMessage(Encoding.UTF8.GetString(MTOMInMemory.ToArray()));
File.WriteAllBytes(v.Filename,v.File);
}
public static TestAttachment DeserializeMTOMMessage(string MTOMMessage)
{
try
{
MemoryStream MTOMMessageInMemory = new MemoryStream(UTF8Encoding.UTF8.GetBytes(MTOMMessage));
XmlDictionaryReader TR = XmlDictionaryReader.CreateMtomReader(MTOMMessageInMemory, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);
DataContractSerializer DCS = new DataContractSerializer(typeof(TestAttachment));
return (TestAttachment)DCS.ReadObject(TR);
}
catch
{
return null;
}
}
}
I would be grateful if someone can help me in pointing out where the problem is. I am new to XOP/MTOM and find it hard to track down where the error might be. Either serialization or de-serialization.
Thank you
There's a bug in your code.
Change your method call
MTOMInMemory.Position = 0;
DeserializeMTOMMessage(Encoding.UTF8.GetString(MTOMInMemory.ToArray()));
to
DeserializeMTOMMessage(MTOMInMemory.ToArray())
and the implementation to
public static TestAttachment DeserializeMTOMMessage(byte[] MTOMMessage)
{
try
{
MemoryStream MTOMMessageInMemory = new MemoryStream(MTOMMessage);
XmlDictionaryReader TR = XmlDictionaryReader.CreateMtomReader(MTOMMessageInMemory, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);
DataContractSerializer DCS = new DataContractSerializer(typeof(TestAttachment));
return (TestAttachment)DCS.ReadObject(TR);
}
catch
{
return null;
}
}
what you've done was some double conversion from utf8 to byte array and vice versa, which ended up creating not the original byte array you were using
I'm implementing a client-server application, and am looking into various ways to serialize and transmit data. I began working with Xml Serializers, which worked rather well, but generate data slowly, and make large objects, especially when they need to be sent over the net. So I started looking into Protobuf, and protobuf-net.
My problem lies in the fact that protobuf doesn't sent type information with it. With Xml Serializers, I was able to build a wrapper which would send and receive any various (serializable) object over the same stream, since object serialized into Xml contain the type name of the object.
ObjectSocket socket = new ObjectSocket();
socket.AddTypeHandler(typeof(string)); // Tells the socket the types
socket.AddTypeHandler(typeof(int)); // of objects we will want
socket.AddTypeHandler(typeof(bool)); // to send and receive.
socket.AddTypeHandler(typeof(Person)); // When it gets data, it looks for
socket.AddTypeHandler(typeof(Address)); // these types in the Xml, then uses
// the appropriate serializer.
socket.Connect(_host, _port);
socket.Send(new Person() { ... });
socket.Send(new Address() { ... });
...
Object o = socket.Read();
Type oType = o.GetType();
if (oType == typeof(Person))
HandlePerson(o as Person);
else if (oType == typeof(Address))
HandleAddress(o as Address);
...
I've considered a few solutions to this, including creating a master "state" type class, which is the only type of object sent over my socket. This moves away from the functionality I've worked out with Xml Serializers, though, so I'd like to avoid that direction.
The second option would be to wrap protobuf objects in some type of wrapper, which defines the type of object. (This wrapper would also include information such as packet ID, and destination.) It seems silly to use protobuf-net to serialize an object, then stick that stream between Xml tags, but I've considered it. Is there an easy way to get this functionality out of protobuf or protobuf-net?
I've come up with a third solution, and posted it below, but if you have a better one, please post it too!
Information on field bounds bug (using System.String):
Hashing:
protected static int ComputeTypeField(Type type) // System.String
{
byte[] data = ASCIIEncoding.ASCII.GetBytes(type.FullName);
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
return Math.Abs(BitConverter.ToInt32(md5.ComputeHash(data), 0));
}
Serialization:
using (MemoryStream stream = new MemoryStream())
{
Serializer.NonGeneric.SerializeWithLengthPrefix
(stream, o, PrefixStyle.Base128, field); // field = 600542181
byte[] data = stream.ToArray();
_pipe.Write(data, 0, data.Length);
}
Deserializaion:
using (MemoryStream stream = new MemoryStream(_buffer.Peek()))
{
lock (_mapLock)
{
success = Serializer.NonGeneric.TryDeserializeWithLengthPrefix
(stream, PrefixStyle.Base128, field => _mappings[field], out o);
}
if (success)
_buffer.Clear((int)stream.Position);
else
{
int len;
if (Serializer.TryReadLengthPrefix(stream, PrefixStyle.Base128, out len))
_buffer.Clear(len);
}
}
field => _mappings[field] throws a KeyNotFoundException while looking for 63671269.
If I replace ToInt32 with ToInt16 in the hash function, the field value is set to 29723 and it works. It also works if I explicitly define System.String's field to 1. Explicitly defining the field to 600542181 has the same effect as using the hash function to define it. The value of the string being serialized does not change the outcome.
This functionality is actually built in, albeit not obviously.
In this scenario, it is anticipated that you would designate a unique number per message type. The overload you are using passes them all in as "field 1", but there is an overload that lets you include this extra header information (it is still the job of the calling code to decide how to map numbers to types, though). You can then specify different types as different fields is the stream (note: this only works with the base-128 prefix style).
I'll need to double check, but the intention is that something like the following should work:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using ProtoBuf;
static class Program
{
static void Main()
{
using (MemoryStream ms = new MemoryStream())
{
WriteNext(ms, 123);
WriteNext(ms, new Person { Name = "Fred" });
WriteNext(ms, "abc");
ms.Position = 0;
while (ReadNext(ms)) { }
}
}
// *** you need some mechanism to map types to fields
static readonly IDictionary<int, Type> typeLookup = new Dictionary<int, Type>
{
{1, typeof(int)}, {2, typeof(Person)}, {3, typeof(string)}
};
static void WriteNext(Stream stream, object obj) {
Type type = obj.GetType();
int field = typeLookup.Single(pair => pair.Value == type).Key;
Serializer.NonGeneric.SerializeWithLengthPrefix(stream, obj, PrefixStyle.Base128, field);
}
static bool ReadNext(Stream stream)
{
object obj;
if (Serializer.NonGeneric.TryDeserializeWithLengthPrefix(stream, PrefixStyle.Base128, field => typeLookup[field], out obj))
{
Console.WriteLine(obj);
return true;
}
return false;
}
}
[ProtoContract] class Person {
[ProtoMember(1)]public string Name { get; set; }
public override string ToString() { return "Person: " + Name; }
}
Note that this doesn't currently work in the v2 build (since the "WithLengthPrefix" code is incomplete), but I'll go and test it on v1. If it works, I'll all the above scenario to the test suite to ensure it does work in v2.
Edit:
yes, it does work fine on "v1", with output:
123
Person: Fred
abc
I've come up with another solution, but I decided to put it as an answer, instead of in the question, because that makes more sense to me. It's pretty ugly, in my opinion, and I've been warned against using reflection, so please comment on it or provide better answers if you have them. Thanks!
class Program
{
static void Main(string[] args)
{
Person person = new Person
{
Id = 12345,
Name = "Fred",
Address = new Address
{
Line1 = "Flat 1",
Line2 = "The Meadows"
}
};
object value;
using (Stream stream = new MemoryStream())
{
Send<Person>(stream, person);
stream.Position = 0;
value = Read(stream);
person = value as Person;
}
}
static void Send<T>(Stream stream, T value)
{
Header header = new Header()
{
Guid = Guid.NewGuid(),
Type = typeof(T)
};
Serializer.SerializeWithLengthPrefix<Header>(stream, header, PrefixStyle.Base128);
Serializer.SerializeWithLengthPrefix<T>(stream, value, PrefixStyle.Base128);
}
static object Read(Stream stream)
{
Header header;
header = Serializer.DeserializeWithLengthPrefix<Header>
(stream, PrefixStyle.Base128);
MethodInfo m = typeof(Serializer).GetMethod("DeserializeWithLengthPrefix",
new Type[] {typeof(Stream), typeof(PrefixStyle)}).MakeGenericMethod(header.Type);
Object value = m.Invoke(null, new object[] {stream, PrefixStyle.Base128} );
return value;
}
}
[ProtoContract]
class Header
{
public Header() { }
[ProtoMember(1, IsRequired = true)]
public Guid Guid { get; set; }
[ProtoIgnore]
public Type Type { get; set; }
[ProtoMember(2, IsRequired = true)]
public string TypeName
{
get { return this.Type.FullName; }
set { this.Type = Type.GetType(value); }
}
}
[ProtoContract]
class Person {
[ProtoMember(1)]
public int Id { get; set; }
[ProtoMember(2)]
public string Name { get; set; }
[ProtoMember(3)]
public Address Address { get; set; }
}
[ProtoContract]
class Address {
[ProtoMember(1)]
public string Line1 { get; set; }
[ProtoMember(2)]
public string Line2 { get; set; }
}