We Keep Coding

Correct way to serialise and deserialise data?

I have an array of class data which I'm serialising into a byte array then pushing it into a database. This program runs on a scheduled basis during the night. On the other end I have another program which pulls this data out of the database, processes it into a report - or at least that's the plan.
The class is incased in 2 namespaces, first the the application name, the second is just something to hold my structures. Eg below.
namespace FibreTrend
{
namespace Structures
{
[Serializable]
public class Trend
{
public Trend(DateTime date, string ref, int port)
{
Date = date;
Reference = ref;
PortNo = port;
}
public DateTime Date;
public string Reference;
public int PortNo;
}
}
}
{
// Function to take the trendData list, convert it to a byte array
// List<Structures.Trend> trendData;
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream mStream = new MemoryStream())
{
bf.Serialize(mStream, trendData.ToArray());
byte[] b = mStream.ToArray();
// code that pushes the array into the database...
}
}
I have a completely separate application which reads in the data from the database as the byte array. I then go to converting it from the bytes back to my data class.
using (MemoryStream mStream = new MemoryStream())
{
BinaryFormatter binaryFormat = new BinaryFormatter();
mStream.Write(data, 0, data.Length);
mStream.Seek(0, SeekOrigin.Begin);
Structures.Trend[] obj = (Structures.Trend[])binaryFormat.Deserialize(mStream);
}
And here is my error. Its telling me it wants the FibreTrend binary to deserialize the data. Why?? My Trend class is the same size, same data layout, its an exact copy and paste from my other project. Why is it insisting on needing my other binary file in companion. When I do put the binary with it then deserialise it into an object it comes put as a FibreTrend.Structures.Trend[]. I'm obviously not going to include the other binary file with it, I'm also not going to double handle the data converting it to a Report.Structures.Trend[]. Its just a stream of 1s and 0s, why can't I just push it into any class that I deem I want, isn't that the purpose of the cast to tell the compiler how I want the data ordered and structured?

Binary Serialized data stream contains a header with type information in it. You can refer to the Binary Format Data structure here. That's why you are getting that exception about missing assembly.
One way to solve your issue is by implementing a SerializationBinder that overrides the type to be deserialized into at runtime and set Binder property on BinaryFormatter. Here is a very good example.
A preferred solution would be to use alternative serialization formats such as XML, JSON.

Serialize a non-serializable custom object from a nuget package and store it in SQLite database

I'm working with some highly custom objects from a nuget package that I need to store in the local SQLite db using Xamarin. I tried using this code
public static byte[] Serialize(ComplexType c)
{
byte[] arrayData;
using(MemoryStream stream = new MemoryStream())
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize( stream, c );
arrayData = stream.ToArray();
stream.Close();
}
return arrayData;
}
public static ComplexType Deserialize(byte[] arrayData)
{
ComplexType c;
using(MemoryStream stream = new MemoryStream())
{
stream.Write( arrayData, 0, arrayData.Length );
stream.Seek( 0, SeekOrigin.Begin );
BinaryFormatter formatter = new BinaryFormatter();
c = (ComplexType) formatter.Deserialize( stream );
}
return c;
}
to serialize and store the object then retrieve it and deserialize it. It seems like it worked fine to do the serialization but now it turns out that the object is not marked serializable. I can't add the SerializableAttribute property to it because it's from a nuget. I tried editing the object definition in the nuget but of course it's locked.
Is there any easy way to pull off serializing an object not marked serializable or perhaps some other simple method of storing a non-serializable custom object in a SQLite database? Or should I just byte the bullet and write really complicated setters and getters to store all of the custom object's individual custom properties broken down into their individual lists of standard properties and basically deconstruct and reconstruct the object every time it goes to and from the database?
I'm seeing mixed reports on ISerializable and derived classes. Will implementing ISerializable on a derived class from the custom object allow the derived class to be serialized? What about IXMLSerializable? Will either of these work? I just want to get some clarification if possible before I go way down the wrong rabbit hole.

Send object over a TCP connection

I've started windows mobile programming today and I have successfully connected to my server.
The application I am making on Visual Studio is not a universal application, but a Windows Mobile Application.
The API DataWriter is used to write data to an output stream, in the applications scenario the output stream is the socket. I.E:
DataWriter dw = new DataWriter(clientSocket.OutputStream);
One of the methods I have been looking at is WriteBytes and WriteBuffer
(Documentation can be found her for API documentation for DataWriter
.
Which method do I use, and why?
How can I convert this class and sent it to my server using the methods mentioned above.
public class Message
{
public string pas { get; internal set; }
public int type { get; internal set; }
public string us { get; internal set; }#
}
//the below code goes in a seperate function
DataWriter dw = new DataWriter(clientSocket.OutputStream);
Message ms = new Message();
ms.type = 1;
ms.us = usernameTextBox.Text;
ms.pas = usernameTextBox.Text;
//TODO: send ms to the server

Between the two methods, WriteBytes() seems like the simpler approach. WriteBuffer() offers you more control over the output buffer/stream, which you can certainly use if and when you need to. But, for all intents and purposes, if you just need to simply open a connection and send it a byte stream, WriteBytes() does the job.
How can I convert this class and sent it to my server
That's entirely up to you, really. What you have to do is define how you're going to "serialize" your class to transmit over the connection (and thereby have to "deserialize" it when the other code receives the data).
There are a few ways to do that, among many others. A straightforward approach (taken from the top answer on that linked question), would be to use the BinaryFormatter class. Something like this:
var ms = new Message();
ms.type = 1;
ms.us = usernameTextBox.Text;
ms.pas = usernameTextBox.Text;
byte[] serializedMessage;
var formatter = new BinaryFormatter();
using (var stream = new MemoryStream())
{
formatter.Serialize(stream, ms);
serializedMessage = ms.ToArray();
}
// now serializedMessage is a byte array to be sent
Then on the other end you'd need to deserialize it back to an object instance. Which might look something like this:
// assuming you have a variable called serializedMessage as the byte array received
Message ms;
using (var stream = new MemoryStream())
{
var formatter = new BinaryFormatter();
stream.Write(serializedMessage, 0, serializedMessage.Length);
stream.Seek(0, SeekOrigin.Begin);
ms = (Message)formatter.Deserialize(stream);
}
You can of course abstract these behind a simpler interface. Or if you're looking for any kind of human readability in the serialization you might try something like a JSON serializer and directly convert the string to a byte array, etc.
Edit: Note that this is really just an example of one of many ways to "serialize" an object. And, as pointed out by a user in comments below, there could be drawbacks to using this binary serializer.
You can use any serializer, really. You can even make your own. Technically overriding .ToString() to print all the properties and then calling that is a form of serialization. The concept is always the same... Convert the in-memory object to a transmittable piece of data, from which an identical in-memory object can later be built. (Technically, saving to a database is a form of serialization.)

ByteArray class for C#

I'm trying to translate a function from ActionScript 3 into C# .NET.
What I have trouble is how to properly use ByteArrays in C#. In As3 there is a specific Class for it that already has most of the functionality i need, but in C# nothing of that sort seems to exist and I can't wrap my head around it.
This is the As3 function:
private function createBlock(type:uint, tag:uint,data:ByteArray):ByteArray
{
var ba:ByteArray = new ByteArray();
ba.endian = Endian.LITTLE_ENDIAN;
ba.writeUnsignedInt(data.length+16);
ba.writeUnsignedInt(0x00);
ba.writeUnsignedInt(type);
ba.writeUnsignedInt(tag);
data.position = 0;
ba.writeBytes(data);
ba.position = 0;
return ba;
}
But from what I gather, in C# I have to use a normal Array with the byte type, like this
byte[] ba = new byte[length];
Now, I looked into the Encoding Class, the BinaryWriter and BinaryFormatter class and researched if somebody made a Class for ByteArrays, but with no luck.
Can somebody nudge me in the right direction please?

You should be able to do this using a combination of MemoryStream and BinaryWriter:
public static byte[] CreateBlock(uint type, uint tag, byte[] data)
{
using (var memory = new MemoryStream())
{
// We want 'BinaryWriter' to leave 'memory' open, so we need to specify false for the third
// constructor parameter. That means we need to also specify the second parameter, the encoding.
// The default encoding is UTF8, so we specify that here.
var defaultEncoding = new UTF8Encoding(encoderShouldEmitUTF8Identifier:false, throwOnInvalidBytes:true);
using (var writer = new BinaryWriter(memory, defaultEncoding, leaveOpen:true))
{
// There is no Endian - things are always little-endian.
writer.Write((uint)data.Length+16);
writer.Write((uint)0x00);
writer.Write(type);
writer.Write(data);
}
// Note that we must close or flush 'writer' before accessing 'memory', otherwise the bytes written
// to it may not have been transferred to 'memory'.
return memory.ToArray();
}
}
However, note that BinaryWriter always uses little-endian format. If you need to control this, you can use Jon Skeet's EndianBinaryWriter instead.
As an alternative to this approach, you could pass streams around instead of byte arrays (probably using a MemoryStream for implementation), but then you will need to be careful about lifetime management, i.e. who will close/dispose the stream when it's done with? (You might be able to get away with not bothering to close/dispose a memory stream since it uses no unmanaged resources, but that's not entirely satisfactory IMO.)

You want to have a byte stream and then extract the array from it:
using(MemoryStream memory = new MemoryStream())
using(BinaryWriter writer = new BinaryWriter(memory))
{
// write into stream
writer.Write((byte)0); // a byte
writer.Write(0f); // a float
writer.Write("hello"); // a string
return memory.ToArray(); // returns the underlying array
}

Serializing and deserializing a List<List<object>> with BinaryFormatter

Let's say I have
List<object> mainList = new List<object>();
And it contains
List<string> stringList = new List<string();
List<CustomClass> custList = new List<CustomClass>();
mainList.Add(stringList);
mainList.Add(custList);
To serialize
Stream stream;
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, mainList);
To deserialize
Stream stream = (Stream)o;
BinaryFormatter formatter = new BinaryFormatter();
List<object> retrievedList = (List<object>)formatter.Deserialize(stream);
At this point, I receive an error that the stream read (deserialization) reached the end of the stream without retrieving a value.
Do I need to specify something besides...
[Serializable]
public class CustomClass { .... }
in the custom class to make this work? Can I not deserialize a List> that contains different type of object every time?
I tried
IList list = (IList)Activator.CreateInstance(typeof(custClassList[0]))
and tried to send and receive this, but got the same issue.
I can however serialize and deserialize a specified type or List, but I really need it to be dynamic.

Basically, BinaryFormatter is a joke. It works in some cases, but will fail in almost identical scenarios for unknown reasons.
The best and superior alternative to BinaryFormatter is the third-party library protobuf-net (https://github.com/mgravell/protobuf-net), developed by Marc Gravel.
This beauty solved all the problems I was having in one pass. It's much easier to set up and reacts more perfectly to complex, custom classes.
I should also mention that it's faster, in the terms of de/serialization.

In order to fix the issue that causes the error "stream read (deserialization) reached the end of the stream ", the stream position needs to reset to 0 as follows...
stream.Position = 0;
Do I need to specify something besides...
[Serializable] public class CustomClass { .... }
Nope...That should be good for what you are doing.
in the custom class to make this work? Can I not deserialize a List>
that contains different type of object every time?
You should be able to serialize any object.