Is there possible to writte protobuf serialized content directly to SharpZipLib stream? When I try to do this, looks like the provided stream is not filled with the data from protobuf. Later I would need to get back the deserialized entity from provided Zip stream.
My code looks like this:
private byte[] ZipContent(T2 content)
{
const short COMPRESSION_LEVEL = 4; // 0-9
const string ENTRY_NAME = "DefaultEntryName";
byte[] result = null;
if (content == null)
return result;
IStreamSerializerProto<T2> serializer = this.GetSerializer(content.GetType());
using (MemoryStream outputStream = new MemoryStream())
{
using (ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream))
{
zipOutputStream.SetLevel(COMPRESSION_LEVEL);
ZipEntry entry = new ZipEntry(ENTRY_NAME);
entry.DateTime = DateTime.Now;
zipOutputStream.PutNextEntry(entry);
serializer.Serialize(zipOutputStream, content);
}
result = outputStream.ToArray();
}
return result;
}
private class ProtobufStreamSerializer<T3> : IStreamSerializerProto<T3>
{
public ProtobufStreamSerializer()
{
ProtoBuf.Serializer.PrepareSerializer<T3>();
}
public void Serialize(Stream outputStream, T3 content)
{
Serializer.Serialize(outputStream, content);
}
public T3 Deserialize(Stream inputStream)
{
T3 deserializedObj;
using (inputStream)
{
deserializedObj = ProtoBuf.Serializer.Deserialize<T3>(inputStream);
}
return deserializedObj;
}
}
Sample of a class which I'm trying to serialize:
[Serializable]
[ProtoContract]
public class Model
{
[XmlElement("ModelCode")]
[ProtoMember(1)]
public int ModelCode { get; set; }
...
}
This is the problem, I believe (in the original code in the question):
public void Serialize(Stream outputStream, T3 content)
{
using (var stream = new MemoryStream())
{
Serializer.Serialize(stream, content);
}
}
You're completely ignoring outputStream, instead writing the data just to a new MemoryStream which is then ignored.
I suspect you just want:
public void Serialize(Stream outputStream, T3 content)
{
Serializer.Serialize(outputStream, content);
}
I'd also suggest removing the using statement from your Deserialize method: I'd expect the caller to be responsible for disposing of the input stream when they're finished with it. Your method can be simplified to:
public T3 Deserialize(Stream inputStream)
{
return ProtoBuf.Serializer.Deserialize<T3>(inputStream);
}
The code (with the edit pointed out by Jon) looks fine. Here it is working:
static void Main()
{
var obj = new Bar{ X = 123, Y = "abc" };
var wrapper = new Foo<Bar>();
var blob = wrapper.ZipContent(obj);
var clone = wrapper.UnzipContent(blob);
}
where Bar is:
[ProtoContract]
class Bar
{
[ProtoMember(1)]
public int X { get; set; }
[ProtoMember(2)]
public string Y { get; set; }
}
and Foo<T> is your class (I didn't know the name), where I have added:
public T2 UnzipContent(byte[] data)
{
using(var ms = new MemoryStream(data))
using(var zip = new ZipInputStream(ms))
{
var entry = zip.GetNextEntry();
var serializer = this.GetSerializer(typeof(T2));
return serializer.Deserialize(zip);
}
}
Also, note that compression is double-edged. In the example I give above, the underlying size (i.e. if we just write to a MemoryStream) is 7 bytes. ZipInputStream "compresses" this 7 bytes down to 179 bytes. Compression works best on larger objects, usually when there is lots of text content.
Related
public class Program
{
static void Main()
{
var resuult = ReaderHelper.ReadFromFile(#"C:\Users\xalva\source\repos\pokerstars\Aletta.txt");
Console.WriteLine(resuult);
Reader reader = new Reader(resuult, 0);
}
}
static class ReaderHelper
{
public static string ReadFromFile(string filePath)
{
string pokerText;
using (FileStream pokerHand = File.OpenRead(filePath))
{
byte[] array = new byte[pokerHand.Length];
pokerHand.Read(array, 0, array.Length);
pokerText = System.Text.Encoding.Default.GetString(array);
}
return pokerText;
}
public class Reader
{
static int Cursor { get; set; }
static string Buffer { get; set; }
public Reader(string buffer, int cursor)
{
Cursor = cursor;
Buffer = buffer;
}
I have a little project where i have to get some information from text file about draw of poker hands.
In class ReaderHelper I get information from a text file, then I create an object of the Reader class and get null in the Buffer. Why so?(It's not all code)
You can just use File.ReadAllText("[...]") to read text from a file:
static void Main()
{
var result = File.ReadAllText(#"C:\Users\xalva\source\repos\pokerstars\Aletta.txt");
Console.WriteLine(result);
Reader reader = new Reader(result, 0);
}
or you can use a StreamReader object, if you want to use streams, to read text:
public static string ReadFromFile(string filePath)
{
string pokerText;
using (FileStream pokerHand = File.OpenRead(filePath))
using (StreamReader sr = new StreamReader(pokerHand))
{
pokerText = sr.ReadToEnd();
}
return pokerText;
}
I got 2 user-defined classes, called Event and Image, Event has a property stored a list of EventImage, called EventImages. And in Image class, there's a byte[] type property which store the byte[] of one Image file.
Here's the definitions of the 2 classes :
[Serializable]
public Class Event
{
public String Name {get; set;}
public DateTime EventTime { get; set;}
public List<Image> EventImages { get; set; }
...
}
[Serializable]
public Class Image
{
public DateTime ImageTime { get; set;}
public byte[] pData {get; set;}
...
}
Now my question is, I want to serialize my Event object to byte[], and I expect that the whole content of it will be serialize, too, but it seems that I failed.
Here's my code to do Serialization :
public static byte[] ObjectToByteArray(object obj)
{
if (obj == null)
{
return null;
}
else
{
BinaryFormatter bF = new BinaryFormatter();
using (MemoryStream mS = new MemoryStream())
{
bF.Serialize(mS, obj);
return mS.ToArray();
}
}
}
and here's the code for verification :
Console.WriteLine(ObjectToByteArray(Event));
Console.WriteLine(ObjectToByteArray(Event.EventImage));
Console.WriteLine(ObjectToByteArray(Event.EventImages.FirstOrDefault().pData));
and the results are(just assumption value) :
100 200 300
But I expect the result should be 600(100+200+300), 500(200+300) and 300.
So, I think my serialization doesn't really serialize the whole content, it just serialize properties with Basic Types, but without the nested objects, am I right?
I've searched lots of posts, and I found plenty of answers to similar questions mentioned "XML Serialization", but I'm not sure whether its helpful or not. Need I use that or is there any other better way? Thanks in advance!
Quickly made a test to check equality. It passes.
Serializes and deserializes correctly.
Conclusion, don't judge whether something is happening or not until you've tested it.
public static void Run()
{
var i = new Image
{
ImageTime = DateTime.UtcNow,
pData = Guid.NewGuid().ToByteArray()
};
var e = new Event
{
Name = Guid.NewGuid().ToString(),
EventTime = DateTime.UtcNow,
EventImages = new List<Image> {i}
};
var bytes = ObjectToByteArray(e);
var e2 = ObjectFromByteArray(bytes);
Console.WriteLine(e.Equals(e2));
}
public static byte[] ObjectToByteArray(object obj)
{
if (obj == null)
{
return null;
}
var bF = new BinaryFormatter();
using (var mS = new MemoryStream())
{
bF.Serialize(mS, obj);
return mS.ToArray();
}
}
public static object ObjectFromByteArray(byte[] bytes)
{
if (bytes == null)
{
return null;
}
var bF = new BinaryFormatter();
using (var mS = new MemoryStream(bytes))
{
return bF.Deserialize(mS);
}
}
I made a class called MyData and marked it with [Serializable], but it doesn't seem to work...
Here's MyData.cs:
namespace Forms_Bot_Rebuild
{
[Serializable]
public class MyData
{
private String _Email;
private String _Password;
private String _WorldId;
public MyData(string email, string password, string worldid)
{
_Email = email;
_Password = password;
_WorldId = worldid;
}
public string Email
{ get { return _Email; } set { _Email = value; } }
public string Password
{ get { return _Password; } set { _Password = value; } }
public string WorldId
{ get { return _WorldId; } set { _WorldId = value; } }
}
}
Here's Form1.cs:
private void button1_Click(object sender, EventArgs e)
{
MyData data = new MyData(boxEmail.Text, boxPass.Text, boxWorldId.Text);
string objectNode = Serialize.ToJsonString(data);
File.WriteAllText(Application.StartupPath + #"\LoginData.json", objectNode);
}
private void button2_Click(object sender, EventArgs e)
{
object myData;
myData = Serialize.Deserialize<MyData>(new FileStream(Application.StartupPath + #"\LoginData.json", FileMode.Open));
boxEmail.Text = data.Email;
boxPass.Text = data.Password;
boxWorldId.Text = data.WorldId;
}
Here's Serialize.cs:
public class Serialize
{
public static StreamReader ToStreamReader(object o)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer ser = new DataContractJsonSerializer(o.GetType());
ser.WriteObject(stream, o);
stream.Position = 0;
StreamReader sr = new StreamReader(stream);
return sr;
}
public static string ToJsonString(object o)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer ser = new DataContractJsonSerializer(o.GetType());
ser.WriteObject(stream, o);
stream.Position = 0;
StreamReader sr = new StreamReader(stream);
return sr.ReadToEnd();
}
public static object Deserialize<T>(Stream stream)
{
DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(T));
stream.Position = 0;
T t = (T)ser.ReadObject(stream);
return t;
}
}
Also, this code was given to me by a friend, so I asked him. He couldn't understand it too. I asked another friend, who, in my opinion, knew decent C#, but he couldn't fix it too.
An unhandled exception of type 'System.Runtime.Serialization.SerializationException' occurred in System.Runtime.Serialization.dll
Additional information: Expecting element 'root' from namespace ''.. Encountered 'None' with name '', namespace ''.
Any opinions?
You should use
[DataContract]
instead of
[Serializable]
and mark all the properties with
[DataMember]
(but funnily, on my PC works even with Serializable, only it serializes fields instead of properties... Mmmh I see someone else noticed it: http://pietschsoft.com/post/2008/02/27/NET-35-JSON-Serialization-using-the-DataContractJsonSerializer)
Note that Deserialize could have this signature:
public T Deserialize<T>(Stream stream)
and you should close streams after using them:
MyData myData;
using (var fs = new FileStream(Application.StartupPath + #"\LoginData.json""LoginData.json", FileMode.Open))
{
myData = (MyData)Serialize.Deserialize<MyData>(fs);
}
If instead you really want to know what you wrote in the title:
Each serialization class/method look for "its" serialization attributes (at this time there are at least three families of attributes in Microsoft .NET: [Serializable], the [Xml*] family, the [DataContract]. Plus if you use JSON.NET there are its [Json*] attributes. Stop. So depending on the class/method you want to use to serialize your data, you have to know which attributes it will look for.
I'm trying to serialize and then deserialize an object whose class derives from a Dictionary<string,int> with a string member field.
public class TempClass : Dictionary<string, int>
{
public string Version;
public TempClass() { }
}
I've written a unit test to capture the problem that I'm running into: the member field is not set when either serializing or deserializing (to/from a byte[]) with protobuf-net. This test fails in the final Assert when validating that the deserialized Version is properly set. It is always set to null instead of the proper "someVersion".
[TestClass]
public class serializationTest
{
[TestMethod]
public void TestMethod1()
{
string newVersion = "someVersion";
TempClass original = new TempClass()
{
{"a", 2},
{"b", 3},
{"c", 1},
};
original.Version = newVersion;
byte[] serialized = Serialize(original);
TempClass deserialized = Deserialize(serialized);
// Validate
foreach (var pair in original)
{
Assert.IsTrue(deserialized.ContainsKey(pair.Key));
Assert.AreEqual(pair.Value, deserialized[pair.Key]);
}
Assert.AreEqual(newVersion, original.Version, "original mapping version not set correctly");
Assert.AreEqual(newVersion, deserialized.Version, "deserialized version doesn't match");
}
private static TempClass Deserialize(byte[] serialized)
{
TempClass deserialized;
using (MemoryStream ms = new MemoryStream())
{
ms.Write(serialized, 0, serialized.Length);
ms.Position = 0;
deserialized = Serializer.Deserialize<TempClass>(ms);
}
return deserialized;
}
private static byte[] Serialize(TempClass mapping)
{
byte[] serialized;
using (MemoryStream ms = new MemoryStream())
{
Serializer.Serialize(ms, mapping);
serialized = ms.ToArray();
}
return serialized;
}
}
I've already attempted the same work with the BinaryFormatter and also the DataContractSerializer to no avail. Can someone please help me spot where I've goofed that causes this test to fail?
Followup question: If instead I redefine the TempClass like this, the constructor is always called instead of setting the member field properly to the original Version. How can I deserialize without the constructor creating a new Version and instead just copy the original one?
public class TempClass : Dictionary<string, int>
{
public string Version;
public TempClass()
{
Version = DateTime.UtcNow.ToString("s");
}
}
May be this effect related to internal realization of serialization of IDictionary types in protobuf.
You can add version data in dictionary or rewrite yours dto object like this sample.
If you use this data object, it will fix your test:
[ProtoContract]
public class TempClass
{
[ProtoMember(1)]
public Dictionary<string, int> data;
[ProtoMember(2)]
public string Version;
public TempClass() { }
}
The third way is to write your own serialization.
[ProtoContract]
public class TempClass
{
[ProtoMember(1)]
public Dictionary<string, int> data;
[ProtoMember(2)]
public string Version;
public TempClass() { }
}
[TestClass]
public class serializationTest
{
[TestMethod]
public void TestMethod1()
{
string newVersion = "someVersion";
TempClass original = new TempClass()
{
data = new Dictionary<string,int>
{
{"a", 2},
{"b", 3},
{"c", 1},
},
Version = newVersion
};
byte[] serialized = Serialize(original);
TempClass deserialized = Deserialize(serialized);
// Validate
foreach (var pair in original.data)
{
Assert.IsTrue(deserialized.data.ContainsKey(pair.Key));
Assert.AreEqual(pair.Value, deserialized.data[pair.Key]);
}
Assert.AreEqual(newVersion, original.Version, "original mapping version not set correctly");
Assert.AreEqual(newVersion, deserialized.Version, "deserialized version doesn't match");
}
private static TempClass Deserialize(byte[] serialized)
{
TempClass deserialized;
using (MemoryStream ms = new MemoryStream())
{
ms.Write(serialized, 0, serialized.Length);
ms.Position = 0;
deserialized = Serializer.Deserialize<TempClass>(ms);
}
return deserialized;
}
private static byte[] Serialize(TempClass mapping)
{
byte[] serialized;
using (MemoryStream ms = new MemoryStream())
{
Serializer.Serialize(ms, mapping);
serialized = ms.ToArray();
}
return serialized;
}
}
Use [ProtoContract(UseProtoMembersOnly = true, IgnoreListHandling = true)]
instead of [ProtoContract] solves the problem for me.
This will prevent protobuf-net to serialize the class with dictionary rules. (See this)
Here is an example.
[ProtoContract(UseProtoMembersOnly = true, IgnoreListHandling = true)]
public class ProtobufTest<T, P> : IDictionary<T, P>
{
[ProtoMember(1)]
private readonly Dictionary<T, P> _dataset;
[ProtoMember(2)]
public string Name;
private ProtobufTest(Dictionary<T, P> dataset, string name)
{
_dataset = dataset ?? new Dictionary<T, P>();
Name = name;
}
public ProtobufTest(string name) : this(new Dictionary<T, P>(), name) { }
private ProtobufTest() : this(null, string.Empty) {}
//
// IDictionary implementation is omitted.
//
}
Here is an example unit test.
[Test]
public void ProtobufTestNameSerializeDeserialize()
{
ProtobufTest<double, double> t = new ProtobufTest<double, double>("233");
ProtobufTest<double, double> d;
using (MemoryStream ms = new MemoryStream())
{
Serializer.SerializeWithLengthPrefix(ms, t, PrefixStyle.Base128);
ms.Position = 0;
d = Serializer.DeserializeWithLengthPrefix<ProtobufTest<double, double>>(ms, PrefixStyle.Base128);
}
Assert.AreEqual(t.Name, d.Name);
}
(These codes are examples ONLY)
I have been using some code to create MTOM by using code from MSDN.
It seems that there is an error and I cannot understand where the problem lies as one of the users on the forum pointed out that there is an error.
The file (JPEG) data get corrupted after a de-serialization. The complete code is listed below.
public class Post_7cb0ff86_5fe1_4266_afac_bcb91eaca5ec
{
[DataContract()]
public partial class TestAttachment
{
private byte[] fileField;
private string filenameField;
[DataMember()]
public byte[] File
{
get
{
return this.fileField;
}
set
{
this.fileField = value;
}
}
[DataMember()]
public string Filename
{
get
{
return this.filenameField;
}
set
{
this.filenameField = value;
}
}
}
public static void Test()
{
string Filename = "Image.jpg";
byte[] file = File.ReadAllBytes(Filename);
TestAttachment Attachment = new TestAttachment();
Attachment.Filename = Filename;
Attachment.File = file;
MemoryStream MTOMInMemory = new MemoryStream();
XmlDictionaryWriter TW = XmlDictionaryWriter.CreateMtomWriter(MTOMInMemory, Encoding.UTF8, Int32.MaxValue, "");
DataContractSerializer DCS = new DataContractSerializer(Attachment.GetType());
DCS.WriteObject(TW, Attachment);
TW.Flush();
Console.WriteLine(Encoding.UTF8.GetString(MTOMInMemory.ToArray()));
var v = DeserializeMTOMMessage(Encoding.UTF8.GetString(MTOMInMemory.ToArray()));
File.WriteAllBytes(v.Filename,v.File);
}
public static TestAttachment DeserializeMTOMMessage(string MTOMMessage)
{
try
{
MemoryStream MTOMMessageInMemory = new MemoryStream(UTF8Encoding.UTF8.GetBytes(MTOMMessage));
XmlDictionaryReader TR = XmlDictionaryReader.CreateMtomReader(MTOMMessageInMemory, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);
DataContractSerializer DCS = new DataContractSerializer(typeof(TestAttachment));
return (TestAttachment)DCS.ReadObject(TR);
}
catch
{
return null;
}
}
}
I would be grateful if someone can help me in pointing out where the problem is. I am new to XOP/MTOM and find it hard to track down where the error might be. Either serialization or de-serialization.
Thank you
There's a bug in your code.
Change your method call
MTOMInMemory.Position = 0;
DeserializeMTOMMessage(Encoding.UTF8.GetString(MTOMInMemory.ToArray()));
to
DeserializeMTOMMessage(MTOMInMemory.ToArray())
and the implementation to
public static TestAttachment DeserializeMTOMMessage(byte[] MTOMMessage)
{
try
{
MemoryStream MTOMMessageInMemory = new MemoryStream(MTOMMessage);
XmlDictionaryReader TR = XmlDictionaryReader.CreateMtomReader(MTOMMessageInMemory, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);
DataContractSerializer DCS = new DataContractSerializer(typeof(TestAttachment));
return (TestAttachment)DCS.ReadObject(TR);
}
catch
{
return null;
}
}
what you've done was some double conversion from utf8 to byte array and vice versa, which ended up creating not the original byte array you were using