I have an array of class data which I'm serialising into a byte array then pushing it into a database. This program runs on a scheduled basis during the night. On the other end I have another program which pulls this data out of the database, processes it into a report - or at least that's the plan.
The class is incased in 2 namespaces, first the the application name, the second is just something to hold my structures. Eg below.
namespace FibreTrend
{
namespace Structures
{
[Serializable]
public class Trend
{
public Trend(DateTime date, string ref, int port)
{
Date = date;
Reference = ref;
PortNo = port;
}
public DateTime Date;
public string Reference;
public int PortNo;
}
}
}
{
// Function to take the trendData list, convert it to a byte array
// List<Structures.Trend> trendData;
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream mStream = new MemoryStream())
{
bf.Serialize(mStream, trendData.ToArray());
byte[] b = mStream.ToArray();
// code that pushes the array into the database...
}
}
I have a completely separate application which reads in the data from the database as the byte array. I then go to converting it from the bytes back to my data class.
using (MemoryStream mStream = new MemoryStream())
{
BinaryFormatter binaryFormat = new BinaryFormatter();
mStream.Write(data, 0, data.Length);
mStream.Seek(0, SeekOrigin.Begin);
Structures.Trend[] obj = (Structures.Trend[])binaryFormat.Deserialize(mStream);
}
And here is my error. Its telling me it wants the FibreTrend binary to deserialize the data. Why?? My Trend class is the same size, same data layout, its an exact copy and paste from my other project. Why is it insisting on needing my other binary file in companion. When I do put the binary with it then deserialise it into an object it comes put as a FibreTrend.Structures.Trend[]. I'm obviously not going to include the other binary file with it, I'm also not going to double handle the data converting it to a Report.Structures.Trend[]. Its just a stream of 1s and 0s, why can't I just push it into any class that I deem I want, isn't that the purpose of the cast to tell the compiler how I want the data ordered and structured?
Binary Serialized data stream contains a header with type information in it. You can refer to the Binary Format Data structure here. That's why you are getting that exception about missing assembly.
One way to solve your issue is by implementing a SerializationBinder that overrides the type to be deserialized into at runtime and set Binder property on BinaryFormatter. Here is a very good example.
A preferred solution would be to use alternative serialization formats such as XML, JSON.
Related
I've started windows mobile programming today and I have successfully connected to my server.
The application I am making on Visual Studio is not a universal application, but a Windows Mobile Application.
The API DataWriter is used to write data to an output stream, in the applications scenario the output stream is the socket. I.E:
DataWriter dw = new DataWriter(clientSocket.OutputStream);
One of the methods I have been looking at is WriteBytes and WriteBuffer
(Documentation can be found her for API documentation for DataWriter
.
Which method do I use, and why?
How can I convert this class and sent it to my server using the methods mentioned above.
public class Message
{
public string pas { get; internal set; }
public int type { get; internal set; }
public string us { get; internal set; }#
}
//the below code goes in a seperate function
DataWriter dw = new DataWriter(clientSocket.OutputStream);
Message ms = new Message();
ms.type = 1;
ms.us = usernameTextBox.Text;
ms.pas = usernameTextBox.Text;
//TODO: send ms to the server
Between the two methods, WriteBytes() seems like the simpler approach. WriteBuffer() offers you more control over the output buffer/stream, which you can certainly use if and when you need to. But, for all intents and purposes, if you just need to simply open a connection and send it a byte stream, WriteBytes() does the job.
How can I convert this class and sent it to my server
That's entirely up to you, really. What you have to do is define how you're going to "serialize" your class to transmit over the connection (and thereby have to "deserialize" it when the other code receives the data).
There are a few ways to do that, among many others. A straightforward approach (taken from the top answer on that linked question), would be to use the BinaryFormatter class. Something like this:
var ms = new Message();
ms.type = 1;
ms.us = usernameTextBox.Text;
ms.pas = usernameTextBox.Text;
byte[] serializedMessage;
var formatter = new BinaryFormatter();
using (var stream = new MemoryStream())
{
formatter.Serialize(stream, ms);
serializedMessage = ms.ToArray();
}
// now serializedMessage is a byte array to be sent
Then on the other end you'd need to deserialize it back to an object instance. Which might look something like this:
// assuming you have a variable called serializedMessage as the byte array received
Message ms;
using (var stream = new MemoryStream())
{
var formatter = new BinaryFormatter();
stream.Write(serializedMessage, 0, serializedMessage.Length);
stream.Seek(0, SeekOrigin.Begin);
ms = (Message)formatter.Deserialize(stream);
}
You can of course abstract these behind a simpler interface. Or if you're looking for any kind of human readability in the serialization you might try something like a JSON serializer and directly convert the string to a byte array, etc.
Edit: Note that this is really just an example of one of many ways to "serialize" an object. And, as pointed out by a user in comments below, there could be drawbacks to using this binary serializer.
You can use any serializer, really. You can even make your own. Technically overriding .ToString() to print all the properties and then calling that is a form of serialization. The concept is always the same... Convert the in-memory object to a transmittable piece of data, from which an identical in-memory object can later be built. (Technically, saving to a database is a form of serialization.)
I have a server which communicates with Clients using a TCP Sockets .
I want to send an object from one of the client (Sender) to server which sends this object to another client (Receiver )
the object contains fields of different types like this
Class Test {
public string key;
public int id;
public string message;
public Test ()
{
// constructor code
}
}
my question is how to convert the object to array of bytes, and when receive this array of bytes in Receiver how to do the opposite operation (convert from array of bytes to objects)?
You need to serialize your object. There are plenty of ways of doing that in C#.
You can serialize your object to binary bytes, XML or custom forms. If you want binary bytes (apparently that's what you're looking for) you can use BinaryFormatter class.
From the MSDN example:
Test test = new Test();
FileStream fs = new FileStream("output", FileMode.Create);
// Construct a BinaryFormatter and use it to serialize the data to the stream.
BinaryFormatter formatter = new BinaryFormatter();
try
{
formatter.Serialize(fs, test);
}
catch (SerializationException e)
{
Console.WriteLine("Failed to serialize. Reason: " + e.Message);
throw;
}
finally
{
fs.Close();
}
Of course, instead of a FileStream object you will use a socket output stream to send data.
If you are considering multiple platforms, I would suggest you use an XML based serialization so you won't run into issues related to platform endianness (byte order).
Ok, i've seen many similar questions both on here and unity forums asking about converting from one format to another. I've got a (hopefully) simple question that i just can't find the answer for. I'm using the game-center plugin from Prime31 to handle a turn based multi-player game. Inside the plugin for Prime31 they ask you for a byte[] to send to the other players. (State data) So the question is, what would be a good way to convert a List to a byte array AND then convert them back from byte array?
for reference this is as complicated a class as i need, i might need to add more members later, but not any different types. ALSO the list of int's(cards in hand) could easily be 4 separate int if that makes converting the list of PokerPlayers to an byte[] any easier. Also at this list is not a set length but will always be 4-8.
public class PokerPlayer{
public string playerID;
public string alias;
public int wildCard;
public List<int> cardsInHand;
public int chips;
}
I feel like the when i see the answer I'm going to smack myself for not seeing the answer sooner. Any pointers/ links to relevant materials would be sweet, i've searched google for a good 3 hours now with similar (SO similar) but not quite the same questions.
You may want to try serialization.
var binFormatter = new BinaryFormatter();
var mStream = new MemoryStream();
binFormatter.Serialize(mStream, myObjToSerialize);
//This gives you the byte array.
mStream.ToArray();
And then if you want to turn the byte array back into an object:
var mStream = new MemoryStream();
var binFormatter = new BinaryFormatter();
// Where 'objectBytes' is your byte array.
mStream.Write (objectBytes, 0, objectBytes.Length);
mStream.Position = 0;
var myObject = binFormatter.Deserialize(mStream) as YourObjectType;
Update:
Microsoft warns about using BinaryFormatter because it is "insecure and can't be made secure".
Please read aka.ms/binaryformatter for more details.
Preferred alternatives
.NET offers several in-box serializers that can handle untrusted data safely:
XmlSerializer and DataContractSerializer to serialize object graphs into and from XML. Do not confuse DataContractSerializer with NetDataContractSerializer.
BinaryReader and BinaryWriter for XML and JSON.
The System.Text.Json APIs to serialize object graphs into JSON.
BinaryFormatter is now a security risk. If I find a good way to do this without using it I'll be back
https://learn.microsoft.com/en-us/dotnet/core/compatibility/core-libraries/5.0/binaryformatter-serialization-obsolete
Edit:
This is still the top result in Google so I'll show what I've done to move away from BinaryFormatter
You need Newtonsoft.Json
public static class ExtendedSerializerExtensions
{
private static readonly JsonSerializerSettings SerializerSettings = new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
};
public static byte[] Serialize<T>(this T source)
{
var asString = JsonConvert.SerializeObject(source, SerializerSettings);
return Encoding.Unicode.GetBytes(asString);
}
public static T Deserialize<T>(this byte[] source)
{
var asString = Encoding.Unicode.GetString(source);
return JsonConvert.DeserializeObject<T>(asString);
}
}
It's not very far to go from here if you need a stream rather than a byte array
Converting data into byte stream (and back) is called serialization (and deserialization).
You can use the BinaryFormatter class to do so.
XmlSerializer formatter = new XmlSerializer(typeof(List<Objects.PIP>));
**MemoryStream stream = new MemoryStream(new byte[1024]);**
formatter.Serialize(stream, Repository.GlobalRepository.PIPInformation);
byte[] bt = stream.ToArray();
foreach (Communication.Client Client in server.ClientList)
{
Client.SendMessage(bt);
}
stream.Flush();
I get an error in stared line as:
There was an error generating the XML document.
InnerException {"Memory stream is not expandable."}
PIP Class
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using ObjectLibrary.Model;
using System.Xml.Serialization;
namespace PIPServer.Objects
{
[XmlRoot(Namespace = "urn:my-namespace")]
public class PIP
{
private long pipID;
public long PipID
{
get { return pipID; }
set { pipID = value; }
}
private CurrencyPair currencyPair;
public CurrencyPair CurrencyPair
{
get { return currencyPair; }
set { currencyPair = value; }
}
...............
}
}
What am i doing wrong?
Also is my method to convert Stream to byte array correct? [Answered]
The MemoryStream has a ToArray() method that you can use. It returns a byte array with the entire contents of the stream. Just make sure you are finished writing to the stream before reading the contents. :-)
As for the error, it is hard to answer without more information. Have you verified that all the objects you try to serialize are valid?
Edit:
From your comments I think I know what the problem is. You define the MemoryStream with a fixed size of 1024 bytes. When you serialize your object the generated xml is larger than that and you end up with an error saying you cannot expand the memory stream. Try to create it without setting an explicit size (just use
new MemoryStream()
), or set a larger size.
Also, if you don't want to serialize a property, just tag it with the XmlIgnore attribute.
With your update to include the inner exception, it becomes clear: since you are initializing the MemoryStream with a fixed-size byte array, the entire XML must fit in your 1024 bytes. It sounds like you have enough properties to make that a problem.
Did you really mean for a maximum of 1024 bytes, or were did you intend to set an initial size capacity for the stream. The latter would be:
MemoryStream stream = new MemoryStream(1024);
At the moment I am experiencing a very frustrating problem. I will try to abstract the problem to make it a bit easier. It has to with serializing my custom object to a database in one process and deserializing it in another process.
I have two assemlies; AppToDB.dll and AppFromDB.dll. I have a 3rd assembly - MyCustomObject.dll - which both of these assemblies contain a reference to. The MyCustomObject.dll extends MarshalByRefObject.
In my AppToDB.dll I execute the following code:
public bool serializeToDB(MyCustomObject obj)
{
OdbcDataAdapter da = new OdbcDataAdapter();
MemoryStream memStream = new MemoryStream();
try
{
ObjRef marshalledObj = RemotingServices.Marshal((System.MarshalByRefObject)obj);
// Serialize the object; construct the desired formatter
IFormatter oBFormatter = new BinaryFormatter();
// Try to serialize the object
oBFormatter.Serialize(memStream, marshalledObj);
// Create byte array
byte[] serialized = memStream.ToArray();
// Build the query to write to the database
string queryString =
"INSERT INTO MyCustomObject(id, object) VALUES(?, ?)";
OdbcCommand command = new OdbcCommand(queryString, connection);
command.Parameters.AddWithValue("id", 1);
command.Parameters.AddWithValue("object", serialized);
// Write the object byte array to the database
int num = command.ExecuteNonQuery();
}
catch { }
}
In AppFromDB.dll I execute this code:
public OCR.Batch deserializeFromDB()
{
MemoryStream memStream = new MemoryStream();
try
{
string queryString = "SELECT object FROM FCBatch";
OdbcCommand command = new OdbcCommand(queryString, connection);
OdbcDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess);
// Size of the BLOB buffer.
int bufferSize = 100;
// The BLOB byte[] buffer to be filled by GetBytes.
byte[] outByte = new byte[bufferSize];
// The bytes returned from GetBytes.
long retval;
// The starting position in the BLOB output.
long startIndex = 0;
MemoryStream dbStream = new MemoryStream();
while (reader.Read())
{
// Reset the starting byte for the new BLOB.
startIndex = 0;
// Read bytes into outByte[] and retain the number of bytes returned.
retval = reader.GetBytes(0, startIndex, outByte, 0, bufferSize);
// Continue while there are bytes beyond the size of the buffer.
while (retval == bufferSize)
{
dbStream.Write(outByte, 0, bufferSize);
dbStream.Flush();
// Reposition start index to end of last buffer and fill buffer.
startIndex += bufferSize;
retval = reader.GetBytes(0, startIndex, outByte, 0, bufferSize);
}
// Write the remaining buffer.
dbStream.Write(outByte, 0, (int)retval);
dbStream.Flush();
}
// Close the reader and the connection.
reader.Close();
dbStream.Position = 0;
object temp = oBFormatter.Deserialize(dbStream);
MyCustomObject obj = (MyCustomObject)temp;
return null;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return null;
}
}
OK, so in both pieces of code you can see a MemoryStream object. In the first AppToDB it is created and if I look at its contents it contains 707 bytes. Fine. I write it to the database and save it there as a BLOB. Now in AppFromDB I retrieve the BLOB and store it in a byte[] array. I write the byte[] array to a MemoryStream again, and see that my MemoryStream objects contains 707 bytes, all of which are in place like the original. It seems I have transferred the object with success!
Now the problem lies with object temp = oBFormatter.Deserialize(dbStream);. As soon as I try to deserialize, my object is a Transparent Proxy and I am unable to cast to MyCustomObject!! How do I get my original object back? How in ##&'s name can I have a MemoryStream object....IN memory...ready to be serialized...and suddenly it's a Transparent Proxy again.
I am at loss. Help is appreciated. I will pray to ##& for the one who has the answer ;)
Edit 1
OK, I must say things are starting to make sense now (although the problem persists). My problem: I have an object (including state) on one side and I need to store it in the database so I can use it days later by another process on the other side.
My object is not serializable, because it wraps a 3rd party object which is not marked as serializable. So my only option seems to be to marshal, which returns an ObjRef, which in turn IS serializable. But of course - days later - the object I am deserializing is merely the reference and my original object is gone.
How do I solve my problem? More people must have encountered this and I just can't seem to find the answer...
Edit 2
OK, I guess I am going to write my own serializable class isomorphic to the 3rd party object. Then run through the whole 3rd party object and, store/wrap its state info etc. Then serialize my object to the database...Seems to be my only option.
Edit 3
Starting on this problem again after a while. Just realized though that solution posted in Edit 2 won't work. I have to deserialize into an object the 3rd party assembly knows, since it will continue to perform operations on it.
You're serializing the ObjRef which is not the original object but rather a "remoting reference", containing all the information to transfer the reference.
From MSDN (emphasis mine):
The ObjRef contains information that
describes the Type and class of the
object being marshaled, its exact
location, and communication-related
information on how to reach the
remoting subdivision where the object
is located.
After a class implementing
MarshalByRefObject is marshaled, the
ObjRef that represents it is
transferred through a channel into
another application domain, possibly
in another process or computer. When
the ObjRef is deserialized in the
target application domain, it is
parsed to create a transparent proxy
for the remote MBR object. This
operation is known as unmarshaling.
Edit (because of to the new information provided):
There are several approaches possible to solve your problem. All of them have their own set of limitations, which I'll put together here:
Use XML serialization with the XmlSerializer. The XML serialization is different in that it serializes and deserializes all public properties of an object instead of serializing fields and/or custom data (when ISerializable is implemented by the object to be serialized). Therefore, if the 3rd-party objects are merely data containers, serializing the public properties is good enough and they provide a default constructor, you should be fine with this solution.
Build your own "low-level" serialization code, using reflection to get the fields and serialize those. This approach typically requires your application to run with full trust (reflecting and modifying private fields via reflection requires permissions which are typically not present in lower trusts). Some methods which may assist you here are the following: FormatterServices.GetSerializableMembers() to get the fields to serialize, FormatterServices.GetObjectData() to get the field data from an object instance, FormatterServices.GetSafeUninitializedObject() when deseializing to create a new, uninitialized instance (no constructor is called), and FormatterServices.PopulateObjectMembers() to write the fields back to the new instane. If you have only simple data types which are serializable in the fields, you can serialize and deserialize the object[] which you use to store the field data.
Your current idea, which is to manually write a replica of the 3rd-party objects. This can be very painful and basically only works when also XML serialization would work. If properties are read-only for sinatce, you'll not get far with this approach.