I am writing my custom serializer for my class which should serialize a class object to a byte array. Below is my sample code:
[StructLayout(LayoutKind.Sequential)]
public class ATTRIBUTE_STR
{
public uint atType;
public uint atLen;
public byte[] atData = new byte[3];
public byte[] serialize()
{
byte[] temp = new byte[Marshal.SizeOf(this)];
byte[] buffer = BitConverter.GetBytes(atType);
Array.Copy(buffer, 0, temp, 0, buffer.Length);
buffer = null;
buffer = BitConverter.GetBytes(atLen);
Array.Copy(buffer, 0, temp, 4, buffer.Length);
Array.Copy(this.atData, 0, temp, 8, this.atData.Length);
return temp;
}
};
However, the byte array atData because of how its stored in memory with byte alignment and all, it's not getting correctly into the temp byte array. Since the byte arrays have even byte alignment.
How can I serialize this object accounting for the alignment of members in memory?
EDIT: I know there are other options like Marshalling or ProtocolBuffers, but I would like using my custom serializer.
Your sizeof call is probably not returning what you think it is, but there hardly seems much point worrying about Sizeof if you're going to stick with hard-coded offsets for the output.]
So you might as well do something like this:
byte[] temp = new byte[8+atLen.Length];
BitConverter.GetBytes(atType).CopyTo(temp,0);
BitConverter.GetBytes(atLen).CopyTo(temp,4);
this.AtData.CopyTo(temp,8);
return temp;
Update: Sizeof returns the amount of unmanaged space necessary to store your object - but bear in mind that your atData array contents are not stored within your object, they are in an array object which is referred to by your object. So Marshal.Sizeof is not going to give you an answer which includes the size of your array - indeed, if you ask Marshal.Sizeof to give you Sizeof(atData), you'll get an exception.
Related
I have this method to convert long to little endian byte array
public static byte[] UnsignedIntegerToLEByteArray(ulong value)
{
// Value in bytes... in your system's endianness (let's say: little endian)
byte[] bytes = BitConverter.GetBytes(value);
// If it was big endian, reverse it
if (!BitConverter.IsLittleEndian)
Array.Reverse(bytes);
return bytes;
}
My goal is to use it for shorted data types too, like int, short, etc. See here:
byte a = 0xAA;
ushort b = 0xEEAA;
uint c = 0xAABBCCDD;
ulong d = 0xAABBCCDDAAAAAABB;
// If you passed short below, you are only interested
// in first two bytes of the array
byte []tmp = DppUtilities.UnsignedIntegerToLEByteArray(b);
This works if my machine is little endian.
Will it also work if it is run on big endian machine? I think yes, but I would like to verify.
You can play a trick with a help of IntPtr and Marshal in order to convert any struct (including byte, ushort and ulong):
// Disclaimer: the structure will be reversed as a whole, not field by field
public static byte[] ToLEByteArray<T>(T value) where T: struct {
int size = Marshal.SizeOf(typeof(T));
byte[] bytes = new byte[size];
IntPtr p = Marshal.AllocHGlobal(size);
try {
Marshal.StructureToPtr(value, p, true);
Marshal.Copy(p, bytes, 0, size);
}
finally {
Marshal.FreeHGlobal(p);
}
// If it was big endian, reverse it
if (!BitConverter.IsLittleEndian)
Array.Reverse(bytes);
return bytes;
}
....
Byte b = 123;
ushort s = 123;
ulong l = 123;
Byte[] result_byte = ToLEByteArray(b);
Byte[] result_ushort = ToLEByteArray(s);
Byte[] result_ulong = ToLEByteArray(l);
....
int i = 123456;
Byte[] result_int = ToLEByteArray(i);
EDIT: what's wrong with the implementation in the question? (from the comment).
Or, restating the question, what is that stuff with IntPtr, Marshal for?
The main issue of the question's implementation is the initial conversion to ulong:
// all the arguments will be converted to ulong
public static byte[] UnsignedIntegerToLEByteArray(ulong value)
In order to illustrate the problem, imagine, that we have two values
Byte x = 0x12; // 18
ulong u = 0x0000000000000012; // 18
and the expected output is
new byte[] {0x12}; // for a single byte
new byte[] {0x12, 0, 0, 0, 0, 0, 0, 0}; // for 8 bytes, i.e. ulong
however, the actual output will be
new byte[] {0x12, 0, 0, 0, 0, 0, 0, 0};
for both byte and ulong. This misbehaviour can lead to problems if you, say, want to write down numeric values (byte, short, ulong etc.), to a binary file, pass them to a binary stream etc:
using (Stream stm = ...) {
...
Byte[] buffer = UnsignedIntegerToLEByteArray(...);
stm.Write(buffer, offset, buffer.Length); // <- the (possibly!) erroneous write
...
}
I had the same thing and I tested it.
I can tell you it works 100%. If you want to check it out yourself, you can for example just pretend you need it the other way. So you reverse the byte array and reverse it if it's IsLittleEndian.
Your output is always in LittleEndian in your method, which is what you want.
Like BitConverter.GetBytes has different overloads for that, you should do it the same way:
public static byte[] UnsignedIntegerToLEByteArray(ulong value) { ...
public static byte[] UnsignedIntegerToLEByteArray(int value) { ...
public static byte[] UnsignedIntegerToLEByteArray(short value) { ...
The complier chooses the right overload when using it like you already did:
byte []tmp = DppUtilities.UnsignedIntegerToLEByteArray(b);
If you try to do this with a single method that try to check the zero bytes, it will produce wrong results:
How to tell if the number 00 00 12 34 should reverse 2 or 4 bytes?
I have a structure which I want to send to a TCP client throught TCP protocol so I want to assign or copy this struct data to byte array:
struct StartReadXML
{
public int CmdID;//3
public char[] CmdName;//ReadXML
public char[] Description;//Other data
};
here am assigning data to struct data members as below :
StartReadXML startXML=new StartReadXML();
startXML.CmdID = 3;
startXML.CmdName = "sreedhar".ToCharArray();
startXML.Description = "Kumar".ToCharArray();
Now, I want it to be assigned to a byte array. Which am doing using marshalling as below:
int sizestartXML = Marshal.SizeOf(startXML);//Get size of struct data
byte[] startXML_buf = new byte[sizestartXML];//byte array & its size
IntPtr ptr = Marshal.AllocHGlobal(sizestartXML);//pointer to byte array
Marshal.StructureToPtr(startXML, ptr, true);
Marshal.Copy(ptr, startXML_buf, 0, sizestartXML);
Marshal.FreeHGlobal(ptr);
//Sending struct data packet
stm.Write(startXML_buf, 0, startXML_buf.Length);//Modified
But, it fails at Structuretoptr conversion method. Please help in transferring the struct data as bytes for which am using above steps.
Thanks in advance Smile | :) !!
You cannot call StructureToPtr on arrays of variable size.
What this boils down to is that unless you know the size of CmdName and declare it - if it would be for example, 20 chars in size, like so:
public fixed char[] CmdName[20];
You will be greeted with an exception from the Marshal saying that your structure is either non-blittable or no meaningful size can be obtained.
This is a requirement the CLR imposes, and you can not work around.
An alternative method would be to use the Convert class or a serializer to convert the members of your struct manually, but unless you know the size of those arrays up front, you won't be able to use StructureToPtr - the same goes for the string type, as I'm assuming that's what your char array will contain.
Consider using a MemoryStream and writing values to the stream, and sending the contents of the stream using stream.ToArray() instead.
Considering that you can't simply convert to binary a struct, use for example:
class StartReadXML
{
public int CmdID;//3
public string CmdName;//ReadXML
public string Description;//Other data
}
Then:
var srx = new StartReadXML();
srx.CmdID = 3;
srx.CmdName = "sreedhar";
srx.Description = "Kumar";
// Example of how to Write to byte[] buffer
byte[] buffer;
using (var ms = new MemoryStream())
{
using (var bw = new BinaryWriter(ms, Encoding.UTF8))
{
bw.Write(srx.CmdID);
bw.Write(srx.CmdName);
bw.Write(srx.Description);
}
buffer = ms.ToArray();
}
// Example of how to Read from byte[] buffer
var srx2 = new StartReadXML();
using (var ms = new MemoryStream(buffer))
{
using (var br = new BinaryReader(ms, Encoding.UTF8))
{
srx2.CmdID = br.ReadInt32();
srx2.CmdName = br.ReadString();
srx2.Description = br.ReadString();
}
}
Note that here I'm working with a "variable length" packet: the length of CmdName and Description aren't fixed (and BinaryWriter/BinaryReader handle this by prepending the length of the string).
The opposite is when you have a packet of fixed length, with fixed-length strings. That requires a totally different handling.
Some code I'm modifying makes extensive use of structs to communicate with some factory equipment by loading them to or from byte arrays.
Here's a made-up ultra-simplified example of such a struct.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct K_FOO
{
public byte a_byte; // 1 byte
public BYTE3_TYPE b3; // 3 bytes
public int num; // happens to be 4 bytes
}
BYTE3_TYPE looks like this...
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class BYTE3_TYPE
{
[System.Runtime.InteropServices.MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
public byte[] L = new byte[3];
}
If I just do a
K_FOO origFoo = new K_FOO();
the int and the byte are initialized to 0's, I assume because they are native types, but the byte array b3 is *un*initialized - all nulls. I have to explicitly load it, for example,
BYTE3_TYPE b3 = new BYTE3_TYPE();
origFoo.b3 = b3;
... and I couldn't think of any alternative because structs don't take parameterless constructors but the real structs are huge.
But I noticed something interesting. We have routines to copy these structs to and from byte arrays. For example . . .
public static T ByteArrayToStructure<T>(byte[] buffer) where T : struct
{
int length = buffer.Length;
IntPtr ptr = Marshal.AllocHGlobal(length); // allocate (length) bytes
Marshal.Copy(buffer, 0, ptr, length); // copies into UNmanaged space
T result = (T)Marshal.PtrToStructure(ptr, typeof(T));
Marshal.FreeHGlobal(ptr);
return result;
}
...and if I call it...
K_FOO retFoo = ByteArrayToStructure<K_FOO>(buffer.bytes);
... the resulting struct is returned fully initialized, the bytes in the byte array have all had their space allocated so they could be loaded, apparently in the PtrToStructure() call. This implies .Net "knows" how to initialize such a struct. So is there some way to get .Net to do that for me so I can avoid writing hundreds of lines of explicit initialization code? Thanks in advance!
If you make your BYTE3_TYPE a struct instead of a class, the default constructor (calling K_FOO origFoo = new K_FOO();) will initialize the entire thing to zero correctly.
This is also likely the correct approach if you're trying to match existing specifications and pass this to custom hardware.
I have an array of bytes that are actually 16-bit samples from a sound card. So 1000 bytes actually represents 500 Short (16-bit values).
Currently I'm converting them like this:
byte [] inputData = new byte[1000];
short [] convertedData = new short[500];
Buffer.BlockCopy(inputData, 0, convertedData , 0, 1000);
It works fine and it's pretty quick as it's a low-level byte copy.
However is there a way to do it without the copy? i.e. tell C# to treat this area of memory as an array of 500 shorts instead of 1000 bytes? I know that in C/C++ I could just cast the pointer and it would work.
This copy happens in a tight loop, up to 5000 times a second, so if I can remove the copy it would be worthwhile.
StructLayout lets you control the physical layout of the data fields in a class or structure. It is typically used when interfacing with unmanaged code which expects the data in a specific layout.
Give this a try:
[StructLayout(LayoutKind.Explicit)]
struct UnionArray
{
[FieldOffset(0)]
public Byte[] Bytes;
[FieldOffset(0)]
public short[] Shorts;
}
static void Main(string[] args)
{
var union = new UnionArray() {Bytes = new byte[1024]};
foreach (short s in union.Shorts)
{
Console.WriteLine(s);
}
}
Perhaps a C# analog of the C-language union would do the trick.
bytes[] inputData = Array.ConvertAll<short, bytes>(bytes, delegate(short[] convertedData) { return short.Parse(convertedData); } );
something like that, i didnt check, but maybe that'll help you out :)
This is related to my previous question, and thought will make this as a sparate question as it will make more sense.
I have created my Struct as :
public struct Smb_Parameters
{
public byte WordCount;
public ushort[] Words;
}
Without Assigning any Values when I try to get the size of the Struct it returns 4-Bytes:
Smb_Parameters smbParameter = new Smb_Parameters();
int len1 = Marshal.SizeOf(smbParameter );
MessageBox.Show(len1.ToString());
But When I assign the Values to structure fields :
Smb_Parameters smbParameter = new Smb_Parameters();
string myString= "String ll be converted to byte";
smbParameter.WordCount=0x00;
smbParameter .Words=Encoding.ASCII.GetBytes(myString);
int len1 = Marshal.SizeOf(smbParameter );
MessageBox.Show(len1.ToString());
Still now It shows the Length as 4-Bytes but I need the updated size.
If you wish to get the size as if it was an unmanaged type, you'd need to supply some information about its fields (e.g., the length of the array). Without it, the array length would not be taken into account.
e.g.,
public struct Smb_Parameters1
{
public byte WordCount; //1 byte
public ushort[] Words; //4 bytes (a "pointer")
}
Marshal.SizeOf(typeof(Smb_Parameters1)); //8 (with padding)
//I don't see how you get 4 unless you are on a 16-bit system maybe
[StructLayout(LayoutKind.Sequential)]
public struct Smb_Parameters2
{
public byte WordCount; //1 byte
[MarshalAs(UnmanagedType.ByValArray, SizeConst=10)]
public ushort[] Words; //20 bytes (2 * 10 bytes)
}
Marshal.SizeOf(typeof(Smb_Parameters2)); //22 (with padding)
Note that these are sizes are fixed (as if it was declared in a C/C++ program). The size reported by SizeOf() will only use these and not take into account what size array you store in Words.
Smb_Parameters1 s1 = new Smb_Parameters1() { Words = new ushort[] { 0, 1, 2 } };
Smb_Parameters2 s2 = new Smb_Parameters2() { Words = new ushort[] { 0, 1, 2 } };
Marshal.SizeOf(s1); //8 bytes
Marshal.SizeOf(s2); //22 bytes
The Words field is an array, and arrays are reference types. The structure doesn't actually contains the array items, it only contains a reference to the array, which is stored somewhere else (typically on the heap). So SizeOf always returns the same size, since the size of a reference doesn't depend on the size of the object pointed by that reference.
Marshal.SizeOf() returns a fixed size of an class/struct. The length does never depend on the contents of your object passed.
You could calculate the size with Marshal.SizeOf(typeof(byte))+Marshal.SizeOf(typeof(ushort))*yourarraylength
Another way would be to use the BinaryFormatter class to serialize your struct into binary form. It returns you a byte array. If you take the length of it you know its serialized size. Note that the result of BinaryFormatter cannot not be easily read by a non-.net language since its format is native to .net only.