At the moment I am experiencing a very frustrating problem. I will try to abstract the problem to make it a bit easier. It has to with serializing my custom object to a database in one process and deserializing it in another process.
I have two assemlies; AppToDB.dll and AppFromDB.dll. I have a 3rd assembly - MyCustomObject.dll - which both of these assemblies contain a reference to. The MyCustomObject.dll extends MarshalByRefObject.
In my AppToDB.dll I execute the following code:
public bool serializeToDB(MyCustomObject obj)
{
OdbcDataAdapter da = new OdbcDataAdapter();
MemoryStream memStream = new MemoryStream();
try
{
ObjRef marshalledObj = RemotingServices.Marshal((System.MarshalByRefObject)obj);
// Serialize the object; construct the desired formatter
IFormatter oBFormatter = new BinaryFormatter();
// Try to serialize the object
oBFormatter.Serialize(memStream, marshalledObj);
// Create byte array
byte[] serialized = memStream.ToArray();
// Build the query to write to the database
string queryString =
"INSERT INTO MyCustomObject(id, object) VALUES(?, ?)";
OdbcCommand command = new OdbcCommand(queryString, connection);
command.Parameters.AddWithValue("id", 1);
command.Parameters.AddWithValue("object", serialized);
// Write the object byte array to the database
int num = command.ExecuteNonQuery();
}
catch { }
}
In AppFromDB.dll I execute this code:
public OCR.Batch deserializeFromDB()
{
MemoryStream memStream = new MemoryStream();
try
{
string queryString = "SELECT object FROM FCBatch";
OdbcCommand command = new OdbcCommand(queryString, connection);
OdbcDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess);
// Size of the BLOB buffer.
int bufferSize = 100;
// The BLOB byte[] buffer to be filled by GetBytes.
byte[] outByte = new byte[bufferSize];
// The bytes returned from GetBytes.
long retval;
// The starting position in the BLOB output.
long startIndex = 0;
MemoryStream dbStream = new MemoryStream();
while (reader.Read())
{
// Reset the starting byte for the new BLOB.
startIndex = 0;
// Read bytes into outByte[] and retain the number of bytes returned.
retval = reader.GetBytes(0, startIndex, outByte, 0, bufferSize);
// Continue while there are bytes beyond the size of the buffer.
while (retval == bufferSize)
{
dbStream.Write(outByte, 0, bufferSize);
dbStream.Flush();
// Reposition start index to end of last buffer and fill buffer.
startIndex += bufferSize;
retval = reader.GetBytes(0, startIndex, outByte, 0, bufferSize);
}
// Write the remaining buffer.
dbStream.Write(outByte, 0, (int)retval);
dbStream.Flush();
}
// Close the reader and the connection.
reader.Close();
dbStream.Position = 0;
object temp = oBFormatter.Deserialize(dbStream);
MyCustomObject obj = (MyCustomObject)temp;
return null;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return null;
}
}
OK, so in both pieces of code you can see a MemoryStream object. In the first AppToDB it is created and if I look at its contents it contains 707 bytes. Fine. I write it to the database and save it there as a BLOB. Now in AppFromDB I retrieve the BLOB and store it in a byte[] array. I write the byte[] array to a MemoryStream again, and see that my MemoryStream objects contains 707 bytes, all of which are in place like the original. It seems I have transferred the object with success!
Now the problem lies with object temp = oBFormatter.Deserialize(dbStream);. As soon as I try to deserialize, my object is a Transparent Proxy and I am unable to cast to MyCustomObject!! How do I get my original object back? How in ##&'s name can I have a MemoryStream object....IN memory...ready to be serialized...and suddenly it's a Transparent Proxy again.
I am at loss. Help is appreciated. I will pray to ##& for the one who has the answer ;)
Edit 1
OK, I must say things are starting to make sense now (although the problem persists). My problem: I have an object (including state) on one side and I need to store it in the database so I can use it days later by another process on the other side.
My object is not serializable, because it wraps a 3rd party object which is not marked as serializable. So my only option seems to be to marshal, which returns an ObjRef, which in turn IS serializable. But of course - days later - the object I am deserializing is merely the reference and my original object is gone.
How do I solve my problem? More people must have encountered this and I just can't seem to find the answer...
Edit 2
OK, I guess I am going to write my own serializable class isomorphic to the 3rd party object. Then run through the whole 3rd party object and, store/wrap its state info etc. Then serialize my object to the database...Seems to be my only option.
Edit 3
Starting on this problem again after a while. Just realized though that solution posted in Edit 2 won't work. I have to deserialize into an object the 3rd party assembly knows, since it will continue to perform operations on it.
You're serializing the ObjRef which is not the original object but rather a "remoting reference", containing all the information to transfer the reference.
From MSDN (emphasis mine):
The ObjRef contains information that
describes the Type and class of the
object being marshaled, its exact
location, and communication-related
information on how to reach the
remoting subdivision where the object
is located.
After a class implementing
MarshalByRefObject is marshaled, the
ObjRef that represents it is
transferred through a channel into
another application domain, possibly
in another process or computer. When
the ObjRef is deserialized in the
target application domain, it is
parsed to create a transparent proxy
for the remote MBR object. This
operation is known as unmarshaling.
Edit (because of to the new information provided):
There are several approaches possible to solve your problem. All of them have their own set of limitations, which I'll put together here:
Use XML serialization with the XmlSerializer. The XML serialization is different in that it serializes and deserializes all public properties of an object instead of serializing fields and/or custom data (when ISerializable is implemented by the object to be serialized). Therefore, if the 3rd-party objects are merely data containers, serializing the public properties is good enough and they provide a default constructor, you should be fine with this solution.
Build your own "low-level" serialization code, using reflection to get the fields and serialize those. This approach typically requires your application to run with full trust (reflecting and modifying private fields via reflection requires permissions which are typically not present in lower trusts). Some methods which may assist you here are the following: FormatterServices.GetSerializableMembers() to get the fields to serialize, FormatterServices.GetObjectData() to get the field data from an object instance, FormatterServices.GetSafeUninitializedObject() when deseializing to create a new, uninitialized instance (no constructor is called), and FormatterServices.PopulateObjectMembers() to write the fields back to the new instane. If you have only simple data types which are serializable in the fields, you can serialize and deserialize the object[] which you use to store the field data.
Your current idea, which is to manually write a replica of the 3rd-party objects. This can be very painful and basically only works when also XML serialization would work. If properties are read-only for sinatce, you'll not get far with this approach.
Related
I have an array of class data which I'm serialising into a byte array then pushing it into a database. This program runs on a scheduled basis during the night. On the other end I have another program which pulls this data out of the database, processes it into a report - or at least that's the plan.
The class is incased in 2 namespaces, first the the application name, the second is just something to hold my structures. Eg below.
namespace FibreTrend
{
namespace Structures
{
[Serializable]
public class Trend
{
public Trend(DateTime date, string ref, int port)
{
Date = date;
Reference = ref;
PortNo = port;
}
public DateTime Date;
public string Reference;
public int PortNo;
}
}
}
{
// Function to take the trendData list, convert it to a byte array
// List<Structures.Trend> trendData;
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream mStream = new MemoryStream())
{
bf.Serialize(mStream, trendData.ToArray());
byte[] b = mStream.ToArray();
// code that pushes the array into the database...
}
}
I have a completely separate application which reads in the data from the database as the byte array. I then go to converting it from the bytes back to my data class.
using (MemoryStream mStream = new MemoryStream())
{
BinaryFormatter binaryFormat = new BinaryFormatter();
mStream.Write(data, 0, data.Length);
mStream.Seek(0, SeekOrigin.Begin);
Structures.Trend[] obj = (Structures.Trend[])binaryFormat.Deserialize(mStream);
}
And here is my error. Its telling me it wants the FibreTrend binary to deserialize the data. Why?? My Trend class is the same size, same data layout, its an exact copy and paste from my other project. Why is it insisting on needing my other binary file in companion. When I do put the binary with it then deserialise it into an object it comes put as a FibreTrend.Structures.Trend[]. I'm obviously not going to include the other binary file with it, I'm also not going to double handle the data converting it to a Report.Structures.Trend[]. Its just a stream of 1s and 0s, why can't I just push it into any class that I deem I want, isn't that the purpose of the cast to tell the compiler how I want the data ordered and structured?
Binary Serialized data stream contains a header with type information in it. You can refer to the Binary Format Data structure here. That's why you are getting that exception about missing assembly.
One way to solve your issue is by implementing a SerializationBinder that overrides the type to be deserialized into at runtime and set Binder property on BinaryFormatter. Here is a very good example.
A preferred solution would be to use alternative serialization formats such as XML, JSON.
I'm working with some highly custom objects from a nuget package that I need to store in the local SQLite db using Xamarin. I tried using this code
public static byte[] Serialize(ComplexType c)
{
byte[] arrayData;
using(MemoryStream stream = new MemoryStream())
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize( stream, c );
arrayData = stream.ToArray();
stream.Close();
}
return arrayData;
}
public static ComplexType Deserialize(byte[] arrayData)
{
ComplexType c;
using(MemoryStream stream = new MemoryStream())
{
stream.Write( arrayData, 0, arrayData.Length );
stream.Seek( 0, SeekOrigin.Begin );
BinaryFormatter formatter = new BinaryFormatter();
c = (ComplexType) formatter.Deserialize( stream );
}
return c;
}
to serialize and store the object then retrieve it and deserialize it. It seems like it worked fine to do the serialization but now it turns out that the object is not marked serializable. I can't add the SerializableAttribute property to it because it's from a nuget. I tried editing the object definition in the nuget but of course it's locked.
Is there any easy way to pull off serializing an object not marked serializable or perhaps some other simple method of storing a non-serializable custom object in a SQLite database? Or should I just byte the bullet and write really complicated setters and getters to store all of the custom object's individual custom properties broken down into their individual lists of standard properties and basically deconstruct and reconstruct the object every time it goes to and from the database?
I'm seeing mixed reports on ISerializable and derived classes. Will implementing ISerializable on a derived class from the custom object allow the derived class to be serialized? What about IXMLSerializable? Will either of these work? I just want to get some clarification if possible before I go way down the wrong rabbit hole.
I've started windows mobile programming today and I have successfully connected to my server.
The application I am making on Visual Studio is not a universal application, but a Windows Mobile Application.
The API DataWriter is used to write data to an output stream, in the applications scenario the output stream is the socket. I.E:
DataWriter dw = new DataWriter(clientSocket.OutputStream);
One of the methods I have been looking at is WriteBytes and WriteBuffer
(Documentation can be found her for API documentation for DataWriter
.
Which method do I use, and why?
How can I convert this class and sent it to my server using the methods mentioned above.
public class Message
{
public string pas { get; internal set; }
public int type { get; internal set; }
public string us { get; internal set; }#
}
//the below code goes in a seperate function
DataWriter dw = new DataWriter(clientSocket.OutputStream);
Message ms = new Message();
ms.type = 1;
ms.us = usernameTextBox.Text;
ms.pas = usernameTextBox.Text;
//TODO: send ms to the server
Between the two methods, WriteBytes() seems like the simpler approach. WriteBuffer() offers you more control over the output buffer/stream, which you can certainly use if and when you need to. But, for all intents and purposes, if you just need to simply open a connection and send it a byte stream, WriteBytes() does the job.
How can I convert this class and sent it to my server
That's entirely up to you, really. What you have to do is define how you're going to "serialize" your class to transmit over the connection (and thereby have to "deserialize" it when the other code receives the data).
There are a few ways to do that, among many others. A straightforward approach (taken from the top answer on that linked question), would be to use the BinaryFormatter class. Something like this:
var ms = new Message();
ms.type = 1;
ms.us = usernameTextBox.Text;
ms.pas = usernameTextBox.Text;
byte[] serializedMessage;
var formatter = new BinaryFormatter();
using (var stream = new MemoryStream())
{
formatter.Serialize(stream, ms);
serializedMessage = ms.ToArray();
}
// now serializedMessage is a byte array to be sent
Then on the other end you'd need to deserialize it back to an object instance. Which might look something like this:
// assuming you have a variable called serializedMessage as the byte array received
Message ms;
using (var stream = new MemoryStream())
{
var formatter = new BinaryFormatter();
stream.Write(serializedMessage, 0, serializedMessage.Length);
stream.Seek(0, SeekOrigin.Begin);
ms = (Message)formatter.Deserialize(stream);
}
You can of course abstract these behind a simpler interface. Or if you're looking for any kind of human readability in the serialization you might try something like a JSON serializer and directly convert the string to a byte array, etc.
Edit: Note that this is really just an example of one of many ways to "serialize" an object. And, as pointed out by a user in comments below, there could be drawbacks to using this binary serializer.
You can use any serializer, really. You can even make your own. Technically overriding .ToString() to print all the properties and then calling that is a form of serialization. The concept is always the same... Convert the in-memory object to a transmittable piece of data, from which an identical in-memory object can later be built. (Technically, saving to a database is a form of serialization.)
I am using the Kinect Face-Tracking Basics WPF example to collect data from an object.
I can access the data in a text file etc but I want to write the data to managed memory in a C# process and have labview's .NET program pick up the data from same location.
So far, this is what I've got:
this.facePoints3D = frame.Get3DShape();
// using (MemoryStream stream = new MemoryStream())
// {
//var sw = new StreamWriter(stream);
int n = 121;
foreach (Vector3DF[] vector in facePoints3D.GetSlices(n))
{
//convert from float to byte array before we pass on to memory
var bytearray = new byte[vector.Length * this.facePoints3D.Count];
Buffer.BlockCopy(vector, 0, bytearray, 0, bytearray.Length);
//Initialize unmanaged memory to hold array.
int size = Marshal.SizeOf(bytearray[0]) * bytearray.Length;
IntPtr pnt = Marshal.AllocHGlobal(size);
try
{
//copy the array to unmanaged memory.
Marshal.Copy(bytearray, 0, pnt, bytearray.Length);
// Copy the unmanaged array back to another managed array.
byte[] bytearray2 = new byte[bytearray.Length];
Marshal.Copy(pnt, bytearray2, 0, bytearray.Length);
//Console.WriteLine("The array was coppied to unmanaged memory and back.");
}
finally
{
// Free the unmanaged memory.
Marshal.FreeHGlobal(pnt);
}
}
Thus far, I have the labview program configured correctly as
Process A (c#): MemoryStream Buffer -> Marshal AllocHGlobal -> Marshal Copy - > Marshal Dispose -> IntPtr ToInt64
Process B (labview): IntPtr value -> Marshall AllocHGlobal -> Marshal Copy -> Destination
Now the labview end runs well but it doesn't appear to be picking up the values from the Memory location.
Advice please?
Are those 2 separate processes? (2 separate exes). If so, you are not going to be able to share memory via direct allocation due to Process Isolation (1 process cannot see another process's memory).
Assuming that your answer is "yes" (2 separate processes), consider using Named Pipes to communicate cross process (or wrap it up with WCF)
Interprocess communication with Named Pipes: http://msdn.microsoft.com/en-us/library/bb546085(v=vs.110).aspx
WCF Tutorial: basic interprocess communication: http://tech.pro/tutorial/855/wcf-tutorial-basic-interprocess-communication
You could also use a memory-mapped file: http://www.abhisheksur.com/2012/02/inter-process-communication-using.html
EDIT
Added an example of using the binary serializer instead of doing manual memory copies of the structure. You can write the resulting byte arrays to your memory mapped file. This solution does require that you apply the [Serializable] attribute to your Vector3DF structure, and it assumes that the code reading the memory has the same type definition for Vector3DF.
(note: in your code in the comments, it looked as though you are working with an array of arrays of Vector3DF structs, so that's how I modeled the serialization code. Adjust as necessary
public byte[] SerializeVectors(Vector3DF[][] vectors)
{
var formatter = new BinaryFormatter();
// note: if you are using a stream to write to the memory mapped file,
// you could pass it in instead of using this memory stream as an intermediary
using (var stream = new MemoryStream())
{
formatter.Serialize(stream, vectors);
return stream.ToArray();
}
}
public Vector3DF[][] DeserializeVectors(byte[] vectorBuffer)
{
var formatter = new BinaryFormatter();
using (var stream = new MemoryStream(vectorBuffer, false))
{
return (Vector3DF[][])formatter.Deserialize(stream);
}
}
Here is a link to a Gist that contains working code and a unit test so that you can play around with it: https://gist.github.com/jsmarsch/d0dcade8c656b94f5c1c
I'm trying to translate a function from ActionScript 3 into C# .NET.
What I have trouble is how to properly use ByteArrays in C#. In As3 there is a specific Class for it that already has most of the functionality i need, but in C# nothing of that sort seems to exist and I can't wrap my head around it.
This is the As3 function:
private function createBlock(type:uint, tag:uint,data:ByteArray):ByteArray
{
var ba:ByteArray = new ByteArray();
ba.endian = Endian.LITTLE_ENDIAN;
ba.writeUnsignedInt(data.length+16);
ba.writeUnsignedInt(0x00);
ba.writeUnsignedInt(type);
ba.writeUnsignedInt(tag);
data.position = 0;
ba.writeBytes(data);
ba.position = 0;
return ba;
}
But from what I gather, in C# I have to use a normal Array with the byte type, like this
byte[] ba = new byte[length];
Now, I looked into the Encoding Class, the BinaryWriter and BinaryFormatter class and researched if somebody made a Class for ByteArrays, but with no luck.
Can somebody nudge me in the right direction please?
You should be able to do this using a combination of MemoryStream and BinaryWriter:
public static byte[] CreateBlock(uint type, uint tag, byte[] data)
{
using (var memory = new MemoryStream())
{
// We want 'BinaryWriter' to leave 'memory' open, so we need to specify false for the third
// constructor parameter. That means we need to also specify the second parameter, the encoding.
// The default encoding is UTF8, so we specify that here.
var defaultEncoding = new UTF8Encoding(encoderShouldEmitUTF8Identifier:false, throwOnInvalidBytes:true);
using (var writer = new BinaryWriter(memory, defaultEncoding, leaveOpen:true))
{
// There is no Endian - things are always little-endian.
writer.Write((uint)data.Length+16);
writer.Write((uint)0x00);
writer.Write(type);
writer.Write(data);
}
// Note that we must close or flush 'writer' before accessing 'memory', otherwise the bytes written
// to it may not have been transferred to 'memory'.
return memory.ToArray();
}
}
However, note that BinaryWriter always uses little-endian format. If you need to control this, you can use Jon Skeet's EndianBinaryWriter instead.
As an alternative to this approach, you could pass streams around instead of byte arrays (probably using a MemoryStream for implementation), but then you will need to be careful about lifetime management, i.e. who will close/dispose the stream when it's done with? (You might be able to get away with not bothering to close/dispose a memory stream since it uses no unmanaged resources, but that's not entirely satisfactory IMO.)
You want to have a byte stream and then extract the array from it:
using(MemoryStream memory = new MemoryStream())
using(BinaryWriter writer = new BinaryWriter(memory))
{
// write into stream
writer.Write((byte)0); // a byte
writer.Write(0f); // a float
writer.Write("hello"); // a string
return memory.ToArray(); // returns the underlying array
}