Marshal.SizeOf Giving The Wrong Size - c#

The code below is giving the wrong size when using Marshal.SizeOf, but I'm not sure why.
Here is the Struct I'm trying to get the size of:
//[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
//[Serializable]
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct BLEGenericMsg
{
public BLEMessageHdr msg_hdr;
public byte[] msg_body;
public BLEGenericMsg(int messageSize)
{
msg_hdr = new BLEMessageHdr();
msg_body = new byte[messageSize];
}
};
Here is the code that populates the struct and calls the serialize function:
BLEGenericMsg hostKeyMsg = new BLEGenericMsg(serializedPublicBytes.Length);
hostKeyMsg.msg_hdr.msg_id = MESSAGE_BASE_EVENT + EVENT_HOST_PUBLIC_KEY;
hostKeyMsg.msg_body = serializedPublicBytes;
//Only get the size of the body for the entire message, not counter or header
hostKeyMsg.msg_hdr.msg_body_sz = (uint)hostKeyMsg.msg_body.Length;
BluetoothLEHardwareInterface.Log("public Key Size: " + hostKeyMsg.msg_hdr.msg_body_sz + "\n");
byte[] temp = Serialize(hostKeyMsg);
BluetoothLEHardwareInterface.Log("temp Size: " + (uint)temp.Length + "\n");
Here is the serialize function that is getting the size of the struct:
public static byte[] Serialize<T>(T s)
where T : struct
{
var size = Marshal.SizeOf(typeof(T));
BluetoothLEHardwareInterface.Log("BLEGenericMsg Size: " + size + "\n");
var array = new byte[size];
var ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(s, ptr, true);
Marshal.Copy(ptr, array, 0, size);
Marshal.FreeHGlobal(ptr);
return array;
}
The size of serializedPublicBytes is 91 bytes,
the rest of the struct is 6 bytes.
So I'm expecting the Marshal.SizeOf to be 97 bytes,
but instead it is showing only about 14 or 16 bytes.
I tried giving the size of msg_body at instantiation, but that didn't make a difference.
What am I missing?
**edit Here is here is the BLEMessageHdr struct:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct BLEMessageHdr
{
public ushort msg_id;
public uint msg_body_sz;
};

The Marshal.SizeOf() method is not returning the wrong size. In the structure you define:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct BLEGenericMsg
{
public BLEMessageHdr msg_hdr;
public byte[] msg_body;
public BLEGenericMsg(int messageSize)
{
msg_hdr = new BLEMessageHdr();
msg_body = new byte[messageSize];
}
};
the msg_body member is known as a "Flexible Array Member" (FAM) in C. It is an illegal construct in C++. Because it is illegal in C++, and because of the inherent uncertainties in the C standard (§ 6.7.2.1) with regard to the instantiation of a struct that contains a FAM, the Marshal class simply does not accept them for interop with unmanaged code.
The way array members are usually marshalled is with the MarshalAsAttribute, like so:
[MarshalAs(UnmanagedType.ByValArray, SizeConst=N)]
public byte[] msg_body;
where "N" represents the explicitly declared size of the array. Without this attribute, the msg_body member is treated as a pointer by the Marshal class. So, the size that Marshal.SizeOf() is returning is correct. Your generic Serialize() method won't work for structs that have a FAM.
You could modify it to copy over the contents of the FAM manually after the rest has been copied by the Marshal class, but this seems like a rather awkward approach to binary serialization for a managed struct.
// specify the name of the FAM and use reflection to get the value
// THIS ASSUMES that the FAM is always a byte[]
public static byte[] Serialize<T>(T s, string fam) where T : struct
{
Type tt = typeof(T);
// Reflection will get you the bytes in the FAM
FieldInfo fi = tt.GetField(fam);
byte[] famBytes = (byte[])fi.GetValue(s);
// Get the field offset that corresponds to the unmanaged layout for
// the FAM, according to the marshaller
int offset = (int)Marshal.OffsetOf(tt, fam);
var size = Marshal.SizeOf(tt) + famBytes.Length;
BluetoothLEHardwareInterface.Log("BLEGenericMsg Size: " + size + "\n");
var array = new byte[size];
var ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(s, ptr, true);
Marshal.Copy(ptr, array, 0, size);
Marshal.FreeHGlobal(ptr);
// Now you're done with the marshalling, just copy over the contents of the
// byte[] to your resulting array, starting at the correct offset
Array.Copy(famBytes, 0, array, offset, famBytes.Length);
return array;
}
Naturally, you will have to likewise modify the Deserialize() method to deal with structs that have a FAM.
AGAIN, this seems like an awkward approach to this problem. You may want to really reconsider this approach.

Related

how to transform byte[] to struct (contains a byte[] member and length member)

i have a struct define as:
[StructLayout(LayoutKind.Sequential,CharSet = CharSet.Ansi,Pack = 1)]
internal struct Message
{
[MarshalAs(UnmanagedType.U1, SizeConst = 1)]
public byte age;
[MarshalAs(UnmanagedType.U2, SizeConst = 2)]
public ushort length;
[MarshalAs(UnmanagedType.ByValArray, ArraySubType = UnmanagedType.U1,SizeConst = 502)]
public byte[] data;
}
payload received via udp,when received byte[],need to be converted to struct.
data length specified as 502,but actually it should be the length member value indicating the data length,if remove the SizeConst attr, the code will throw Marshal exception at Marshal.SizeOf(typeof(T)).
public static T ToStruct<T>(this byte[] buf)
{
var lth = Marshal.SizeOf(typeof(T));
if (lth > buf.Length) return default(T);
var ptr = Marshal.AllocHGlobal(lth);
Marshal.Copy(buf, 0, ptr, lth);
var structure = Marshal.PtrToStructure(ptr, typeof(T));
Marshal.FreeHGlobal(ptr);
return (T)structure;
}
exception info:
System.ArgumentException: Type 'Itms.Agent.IotBox.TieTa.Entity.Message' cannot be marshaled as an unmanaged structure; no meaningful size or offset can be computed.
i know this is very simple to handle in c/c++, but C# doesn't have a similar approach.
any help?
when received byte[],need to be converted to struct. data length specified as 502,but actually it should be the length member value indicating the data length
that's too complex for simple marshal operations. You'll probably have to serialize/deserialize manually, i.e. something like:
byte[] payload = ...
var age = payload[0];
var len = (payload[1] << 8) | payload[2]; // invert depending on endianness
byte[] data = len == 0 ? Array.Empty<byte>() : new byte[len];
Buffer.BlockCopy(payload, 3, data, len);
var msg = new Message(age, len, data);
This would mean you could remove all the attributes, as you're not using any marshalling features.
Also... the length field seems kinda redundant, since it is simply duplicating the array's length.

How to assign (byte* + offset) to byte[]?

I have a struct:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public unsafe struct FieldIndex {
public fixed byte Meta[16];
}
The first byte is TypeCode. The remaining 15 bytes is a UTF8-encoded string.
I'm using the following to fill the nameBuf byte array:
private static string GetMetaName (FieldIndex meta) {
var idx = 0;
var bytesOut = 1;
var nameBuf = new byte[Indexer.MAXFIELDNAMELENGTH]; // 15
while (bytesOut < Indexer.MAXFIELDLENGTH) { // 16
nameBuf[idx] = *(byte*)((byte*)&meta + bytesOut);
++bytesOut;
++idx;
}
//var src = (byte*)&field.meta + 1;
//fixed (byte* b = nameBuf) {
// *((byte*)b) = (byte)src;
//}
return BinaryConverter.ToString(nameBuf, 0, Indexer.MAXFIELDNAMELENGTH, Indexer.DefaultEncoding);
}
In the commented code above, I was trying to accomplish the same task without the while iteration but it does not work (no compile error, just the wrong interpolation). Can I assign nameBuf without the while-loop?
Update
I'd also prefer using the (byte*) rather than Marshal.Copy.
You could try to use the Marshal.Copy static method in unsafe context:
unsafe
{
Marshal.Copy(new IntPtr(meta), nameBuf, 1, nameBuf.Length);
}
There isn't much point in doing this the hard unsafe way when you can do it just easily this way:
[StructLayout(LayoutKind.Sequential)]
struct FieldIndex {
public byte Typecode;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 15)]
private byte[] Utf8string;
public string GetString() {
return Encoding.UTF8.GetString(Utf8string, 0, 15);
}
}
No while-loop, mission accomplished. That pointer is still there and is just as efficient, you just don't see it anymore. If you fret over calling the GetString() method too often then simply declare another public class and keep this one internal.

C# marshaling of a struct with an array

Let's say that I have a struct similar to
public struct MyStruct
{
public float[] a;
}
and that I want to instantiate one such struct with some custom array size (let's say 2 for this example). Then I marshal it into a byte array.
MyStruct s = new MyStruct();
s.a = new float[2];
s.a[0] = 1.0f;
s.a[1] = 2.0f;
byte[] buffer = new byte[Marshal.SizeOf(typeof(MyStruct))];
GCHandle gcHandle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
try
{
Marshal.StructureToPtr(s, gcHandle.AddrOfPinnedObject(), false);
for (int i = 0; i < buffer.Length; i++)
{
System.Console.WriteLine(buffer[i].ToString("x2"));
}
}
finally
{
gcHandle.Free();
}
This gives me only 4 bytes in my byte[] and they look like a pointer value rather than the value of either 1.0f or 2.0f. I've searched around for ways to make this work, but all I've been able to find so far are similar examples where the struct array size is known ahead of time. Isn't there a way to do this?
The StructureToPtr does only work with structures which contains value types only (int, char, float, other structs). float[] is a reference type and so you really get a kind of pointer (you can't really use it because it is a managed pointer). If you want to copy your array to a pinned memory you have to use one of the Marshal.Copy functions directly on your s.a float array.
Something like that. (I didn't really test it)
byte[] buffer = new byte[sizeof(float) * s.a.Length];
GCHandle gcHandle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
and then
Marshal.Copy(s.a, 0, gcHandle.AddrOfPinnedObject(), s.a.Length);
Update
I have to correct myself. You can get also get what you want by declaring your struct in this way:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 2)]
public float[] a;
}
As you see you have to fix the size of the float array at design time.
There's no straight-forward support in P/Invoke to enable your expected scenario. What I've coded is a technique similar to the one used for storing the Variable length array in a struct in C.
[StructLayout(LayoutKind.Sequential)]
public struct VarLenStruct
{
public int elementCount;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
public float[] array;
public static GCHandle GetUnmanagedStruct(VarLenStruct managedStruct)
{
if (managedStruct.elementCount < 1)
throw new ArgumentOutOfRangeException("The array size must be non-zero");
// Array is a contiguous data structure, so assign the memory appended to the struct for the elements other than the first
int managedBufferSize = Marshal.SizeOf(managedStruct) + Marshal.SizeOf(typeof(float)) * (managedStruct.elementCount - 1);
byte[] managedBuffer = new byte[managedBufferSize];
var handle = GCHandle.Alloc(managedBuffer, GCHandleType.Pinned);
try
{
IntPtr unmgdStructPtr = handle.AddrOfPinnedObject();
Marshal.StructureToPtr(managedStruct, unmgdStructPtr, fDeleteOld: false);
IntPtr unmgdArrAddr = unmgdStructPtr + Marshal.OffsetOf(typeof(VarLenStruct), "array").ToInt32();
Marshal.Copy(source: managedStruct.array, startIndex: 0, destination: unmgdArrAddr, length: managedStruct.elementCount);
}
catch
{
// The handle must be freed in case of any error, since it won't be visible outside this method in this case
handle.Free();
throw;
}
return handle; // Make sure to free this handle at the end of usage!
}
public static VarLenStruct GetManagedStruct(IntPtr unmanagedStructPtr)
{
VarLenStruct resultStruct = (VarLenStruct)Marshal.PtrToStructure(unmanagedStructPtr, typeof(VarLenStruct));
if (resultStruct.elementCount < 1)
throw new NotSupportedException("The array size must be non-zero");
Array.Resize(ref resultStruct.array, newSize: resultStruct.elementCount); // Since the above unmarshalling always gives us an array of Size 1
IntPtr unmgdArrAddr = unmanagedStructPtr + Marshal.OffsetOf(typeof(VarLenStruct), "array").ToInt32();
Marshal.Copy(source: unmgdArrAddr, destination: resultStruct.array, startIndex: 0, length: resultStruct.elementCount);
return resultStruct;
}
}
public static void TestVarLengthArr()
{
VarLenStruct[] structsToTest = new VarLenStruct[]{
new VarLenStruct() { elementCount = 1, array = new float[] { 1.0F } },
new VarLenStruct() { elementCount = 2, array = new float[] { 3.5F, 6.9F } },
new VarLenStruct() { elementCount = 5, array = new float[] { 1.0F, 2.1F, 3.5F, 6.9F, 9.8F } }
};
foreach (var currStruct in structsToTest)
{
var unmgdStructHandle = VarLenStruct.GetUnmanagedStruct(currStruct);
try
{
var ret = VarLenStruct.GetManagedStruct(unmgdStructHandle.AddrOfPinnedObject());
if (!ret.array.SequenceEqual(currStruct.array))
throw new Exception("Code fail!");
}
finally
{
unmgdStructHandle.Free();
}
}
}
I've currently blocked the use of empty array, but with some additional handling you can achieve that too. You can also add validations to your struct constructor in order to check array.Length == elementCount, etc.
Marshal is really for native interop (p/invoke and so on), and native languages don't allow structure members to vary in size at runtime either. (There is a trick called a flexible array member which can appear only at the end... you can deal with that merely by allocating extra memory beyond the structure size).
If you want to serialize an array of primitives to a byte array, just use Buffer.BlockCopy. Arrays of structures of primitives are harder, you can codegen a DynamicMethod which uses the cpblk MSIL instruction or p/invoke memcpy in msvcrt.dll or RtlCopyMemory in kernel32.dll.

How to do memcpy in C# .Net CF with the following task

Hi I am trying to convert the C/C++ Strcut to C# and how to fill the structure member with address of another structure in C#?
C/C++ Struct looks like:
typedef struct _NDISUIO_QUERY_OID
{
NDIS_OID Oid;
PTCHAR ptcDeviceName;
UCHAR Data[sizeof(ULONG)];
} NDISUIO_QUERY_OID, *PNDISUIO_QUERY_OID;
typedef struct My_Struct
{
//les have 2 variables...
UINT a;
UINT b;
}My_STATS, *PMy_STATS;
PNDISUIO_QUERY_OID pQueryOid = NULL;
pQueryOid = (PNDISUIO_QUERY_OID)malloc(sizeof(NDISUIO_QUERY_OID)+ sizeof(My_STATS)) ;
PMy_STATS Statistics;
pQueryOid->Oid = ulOIDCode;//Required OID
pQueryOid->ptcDeviceName = AUB_NAME;//REquired STRING
memcpy(pQueryOid->Data, Statistics, sizeof(My_STATS));
My C# Struct is:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct _NDISUIO_QUERY_OID
{
public uint Oid;
[MarshalAs(UnmanagedType.LPWStr)]
public string ptcDeviceName;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = sizeof(uint))]
public string Data;
};
Problem: How to copy the Statistics structure to Data array in C#??
Thanks :)
Here's my implementation (FYI, the SDF contains all of this code and a lot more)
internal class NDISQueryOid
{
protected const int NDISUIO_QUERY_OID_SIZE = 12;
protected byte[] m_data;
public int Size { get; private set; }
public NDISQueryOid(byte[] data)
{
int extrasize = data.Length;
Size = 8 + extrasize;
m_data = new byte[Size];
Buffer.BlockCopy(data, 0, m_data, DataOffset, data.Length);
}
public NDISQueryOid(int extrasize)
{
Size = NDISUIO_QUERY_OID_SIZE + extrasize;
m_data = new byte[Size];
}
protected const int OidOffset = 0;
public uint Oid
{
get { return BitConverter.ToUInt32(m_data, OidOffset); }
set
{
byte[] bytes = BitConverter.GetBytes(value);
Buffer.BlockCopy(bytes, 0, m_data, OidOffset, 4);
}
}
protected const int ptcDeviceNameOffset = OidOffset + 4;
public unsafe byte* ptcDeviceName
{
get
{
return (byte*)BitConverter.ToUInt32(m_data, ptcDeviceNameOffset);
}
set
{
byte[] bytes = BitConverter.GetBytes((UInt32)value);
Buffer.BlockCopy(bytes, 0, m_data, ptcDeviceNameOffset, 4);
}
}
protected const int DataOffset = ptcDeviceNameOffset + 4;
public byte[] Data
{
get
{
byte[] b = new byte[Size - DataOffset];
Array.Copy(m_data, DataOffset, b, 0, Size - DataOffset);
return b;
}
set
{
Size = 8 + value.Length;
m_data = new byte[Size];
Buffer.BlockCopy(value, 0, m_data, DataOffset, value.Length);
}
}
public byte[] getBytes()
{
return m_data;
}
public static implicit operator byte[](NDISQueryOid qoid)
{
return qoid.m_data;
}
}
Note that in my usage, the NDIS IOCT takes in a pointer (most of my NDIS work is all done as unsafe) so you'd have to do some adjustment there.
So if, for example, you're querying the BSSID, I know the BSSID data is 36 bytes, so I'd create something like this:
var queryOID = new NDISQueryOid(36);
then allocate the name and call NDIS (the production code has a lot more checking than this):
byte[] nameBytes = System.Text.Encoding.Unicode.GetBytes(adapterName + '\0');
fixed (byte* pName = &nameBytes[0])
{
queryOID.ptcDeviceName = pName;
queryOID.Oid = (uint)oid;
var bytes = queryOID.getBytes();
ndis.DeviceIoControl(IOCTL_NDISUIO_QUERY_OID_VALUE, bytes, bytes);
var result = new byte[queryOID.Data.Length];
Buffer.BlockCopy(queryOID.Data, 0, result, 0, result.Length);
}
EDIT
So the result member above is a byte array of the "result" of the query. What it means and how you interpret it depends on what the OID you queried was. For example, if you were querying the currently connected SSID (i.e. NDIS_OID.SSID), then that comes back as a 4-byte length followed by the ASCII-encoded name, so you'd decipher it like this:
int len = BitConverter.ToInt32(data, 0);
if (len > 0)
{
ssid = System.Text.Encoding.ASCII.GetString(data, 4, len);
}
But again, this is only for one specific OID. You have to handle every return case for every incoming OID you decide to support.
First you have the wrong translation of your C++ code: the C# equivalent of a C++ char[] is not a string, it's a byte[]. Once you have that, you just need to know, in general, how to copy a structure into a byte array. Here's a compilable example:
using System;
using System.Runtime.InteropServices;
struct Dest
{
public byte[] Data;
}
struct Src
{
public GCHandle StringHandle;
public long A;
public long B;
}
class Program
{
static void Main()
{
Copy();
}
static void Copy()
{
var str = "Hello";
var src = new Src {
A = 3,
B = 4,
StringHandle = GCHandle.Alloc(str, GCHandleType.Normal)
};
var dst = new Dest();
unsafe
{
Src* srcPtr = &src;
dst.Data = new byte[sizeof(Src)];
Marshal.Copy((IntPtr)srcPtr, dst.Data, 0, sizeof(Src));
}
// When you're sure no one can reference the string anymore
// (Including by accessing the data you put in dst.Data!)
src.StringHandle.Free();
}
EDIT: added example of how to deal with reference types such as strings.
Safely, you can't. .NET enforces type safety, which means that you simply can't force a string to be a structure. However, you can look at the data instead of doing unsafe type casts (why are you storing two uints in a string in the first place? And marshalling it as unicode?
First, you'll have to make Data a byte array. It might be possible to do this with a string as well, but that's just adding encoding issues to the mix; if you can, use byte[] instead. Also, if you don't need to have different kinds of data inside (it seems so), you could simply put the two uint fields right inside the struct and it should work just fine:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct _NDISUIO_QUERY_OID
{
public uint Oid;
[MarshalAs(UnmanagedType.LPWStr)]
public string ptcDeviceName;
public uint DataA;
public uint DataB;
};
The second approach would use a const-sized byte array, long enough to hold the two uints:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct _NDISUIO_QUERY_OID
{
public uint Oid;
[MarshalAs(UnmanagedType.LPWStr)]
public string ptcDeviceName;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = sizeof(ulong))]
public byte[] Data;
};
The first four bytes will be the first uint, the next will be the second.
And of course, you could also use a .NET struct the same way as in the original code - just make sure you use the correct datatype in _NDISUIO_QUERY_OID and it should work automagically.
One point to note though, it seems that the data returned isn't actually necessarily fixed-length. That is quite tricky and it basically means you'd have to deserialize the structure manually based on the pointer and length you get.

Struct to byte array? [duplicate]

How do I convert a structure to a byte array in C#?
I have defined a structure like this:
public struct CIFSPacket
{
public uint protocolIdentifier; //The value must be "0xFF+'SMB'".
public byte command;
public byte errorClass;
public byte reserved;
public ushort error;
public byte flags;
//Here there are 14 bytes of data which is used differently among different dialects.
//I do want the flags2. However, so I'll try parsing them.
public ushort flags2;
public ushort treeId;
public ushort processId;
public ushort userId;
public ushort multiplexId;
//Trans request
public byte wordCount;//Count of parameter words defining the data portion of the packet.
//From here it might be undefined...
public int parametersStartIndex;
public ushort byteCount; //Buffer length
public int bufferStartIndex;
public string Buffer;
}
In my main method, I create an instance of it and assign values to it:
CIFSPacket packet = new CIFSPacket();
packet.protocolIdentifier = 0xff;
packet.command = (byte)CommandTypes.SMB_COM_NEGOTIATE;
packet.errorClass = 0xff;
packet.error = 0;
packet.flags = 0x00;
packet.flags2 = 0x0001;
packet.multiplexId = 22;
packet.wordCount = 0;
packet.byteCount = 119;
packet.Buffer = "NT LM 0.12";
Now I want to send this Packet by socket. For that, I need to convert the structure to a byte array. How can I do it?
My full code is as follows.
static void Main(string[] args)
{
Socket MyPing = new Socket(AddressFamily.InterNetwork,
SocketType.Stream , ProtocolType.Unspecified ) ;
MyPing.Connect("172.24.18.240", 139);
//Fake an IP Address so I can send with SendTo
IPAddress IP = new IPAddress(new byte[] { 172,24,18,240 });
IPEndPoint IPEP = new IPEndPoint(IP, 139);
//Local IP for Receiving
IPEndPoint Local = new IPEndPoint(IPAddress.Any, 0);
EndPoint EP = (EndPoint)Local;
CIFSPacket packet = new CIFSPacket();
packet.protocolIdentifier = 0xff;
packet.command = (byte)CommandTypes.SMB_COM_NEGOTIATE;
packet.errorClass = 0xff;
packet.error = 0;
packet.flags = 0x00;
packet.flags2 = 0x0001;
packet.multiplexId = 22;
packet.wordCount = 0;
packet.byteCount = 119;
packet.Buffer = "NT LM 0.12";
MyPing.SendTo(It takes byte array as parameter);
}
What would a code snippet be?
This is fairly easy, using marshalling.
Top of file
using System.Runtime.InteropServices
Function
byte[] getBytes(CIFSPacket str) {
int size = Marshal.SizeOf(str);
byte[] arr = new byte[size];
IntPtr ptr = IntPtr.Zero;
try
{
ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
}
finally
{
Marshal.FreeHGlobal(ptr);
}
return arr;
}
And to convert it back:
CIFSPacket fromBytes(byte[] arr)
{
CIFSPacket str = new CIFSPacket();
int size = Marshal.SizeOf(str);
IntPtr ptr = IntPtr.Zero;
try
{
ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(arr, 0, ptr, size);
str = (CIFSPacket)Marshal.PtrToStructure(ptr, str.GetType());
}
finally
{
Marshal.FreeHGlobal(ptr);
}
return str;
}
In your structure, you will need to put this before a string
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 100)]
public string Buffer;
And make sure SizeConst is as big as your biggest possible string.
And you should probably read this:
http://msdn.microsoft.com/en-us/library/4ca6d5z7.aspx
If you really want it to be FAST on Windows, you can do it using unsafe code with CopyMemory. CopyMemory is about 5x faster (e.g. 800MB of data takes 3s to copy via marshalling, while only taking .6s to copy via CopyMemory). This method does limit you to using only data which is actually stored in the struct blob itself, e.g. numbers, or fixed length byte arrays.
[DllImport("kernel32.dll", EntryPoint = "CopyMemory", SetLastError = false)]
private static unsafe extern void CopyMemory(void *dest, void *src, int count);
private static unsafe byte[] Serialize(TestStruct[] index)
{
var buffer = new byte[Marshal.SizeOf(typeof(TestStruct)) * index.Length];
fixed (void* d = &buffer[0])
{
fixed (void* s = &index[0])
{
CopyMemory(d, s, buffer.Length);
}
}
return buffer;
}
Have a look at these methods:
byte [] StructureToByteArray(object obj)
{
int len = Marshal.SizeOf(obj);
byte [] arr = new byte[len];
IntPtr ptr = Marshal.AllocHGlobal(len);
Marshal.StructureToPtr(obj, ptr, true);
Marshal.Copy(ptr, arr, 0, len);
Marshal.FreeHGlobal(ptr);
return arr;
}
void ByteArrayToStructure(byte [] bytearray, ref object obj)
{
int len = Marshal.SizeOf(obj);
IntPtr i = Marshal.AllocHGlobal(len);
Marshal.Copy(bytearray,0, i,len);
obj = Marshal.PtrToStructure(i, obj.GetType());
Marshal.FreeHGlobal(i);
}
This is a shameless copy of another thread which I found upon Googling!
Update : For more details, check the source
Variant of the code of Vicent with one less memory allocation:
public static byte[] GetBytes<T>(T str)
{
int size = Marshal.SizeOf(str);
byte[] arr = new byte[size];
GCHandle h = default(GCHandle);
try
{
h = GCHandle.Alloc(arr, GCHandleType.Pinned);
Marshal.StructureToPtr<T>(str, h.AddrOfPinnedObject(), false);
}
finally
{
if (h.IsAllocated)
{
h.Free();
}
}
return arr;
}
public static T FromBytes<T>(byte[] arr) where T : struct
{
T str = default(T);
GCHandle h = default(GCHandle);
try
{
h = GCHandle.Alloc(arr, GCHandleType.Pinned);
str = Marshal.PtrToStructure<T>(h.AddrOfPinnedObject());
}
finally
{
if (h.IsAllocated)
{
h.Free();
}
}
return str;
}
I use GCHandle to "pin" the memory and then I use directly its address with h.AddrOfPinnedObject().
I know this is really late, but with C# 7.3 you can do this for unmanaged structs or anything else that's unmanged (int, bool etc...):
public static unsafe byte[] ConvertToBytes<T>(T value) where T : unmanaged {
byte* pointer = (byte*)&value;
byte[] bytes = new byte[sizeof(T)];
for (int i = 0; i < sizeof(T); i++) {
bytes[i] = pointer[i];
}
return bytes;
}
Then use like this:
struct MyStruct {
public int Value1;
public int Value2;
//.. blah blah blah
}
byte[] bytes = ConvertToBytes(new MyStruct());
As the main answer is using CIFSPacket type, which is not (or no longer) available in C#, I wrote correct methods:
static byte[] getBytes(object str)
{
int size = Marshal.SizeOf(str);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
static T fromBytes<T>(byte[] arr)
{
T str = default(T);
int size = Marshal.SizeOf(str);
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(arr, 0, ptr, size);
str = (T)Marshal.PtrToStructure(ptr, str.GetType());
Marshal.FreeHGlobal(ptr);
return str;
}
Tested, they work.
You can use Marshal (StructureToPtr, ptrToStructure), and Marshal.copy but this is plataform dependent.
Serialization includes Functions to Custom Serialization.
public virtual void GetObjectData(SerializationInfo info, StreamingContext context)
Protected Sub New(ByVal info As SerializationInfo, ByVal context As StreamingContext)
SerializationInfo include functions to serialize each member.
BinaryWriter and BinaryReader also contains methods to Save / Load to Byte Array (Stream).
Note that you can create a MemoryStream from a Byte Array or a Byte Array from a MemoryStream.
You can create a method Save and a method New on your structure:
Save(Bw as BinaryWriter)
New (Br as BinaryReader)
Then you select members to Save / Load to Stream -> Byte Array.
Almost all of the answers here use Marshal.StructureToPtr, which might be good for P/Invoke but it is very slow, and doesn't even always represent the actual raw content of the value. #Varscott128's answer is much better but it also contains an explicit byte copying, which is not necessary.
For unmanaged structs (structs without managed references) all you need is to reinterpret the allocated result array so a simple assignment does the trick (works even for huge structs):
.NET (Core) Solution:
If you can use the Unsafe class, then the solution is really easy. The unsafe modifier is required only due to sizeof(T).
public static unsafe byte[] SerializeValueType<T>(in T value) where T : unmanaged
{
byte[] result = new byte[sizeof(T)];
Unsafe.As<byte, T>(ref result[0]) = value;
return result;
}
// Note: Validation is omitted for simplicity
public static T DeserializeValueType<T>(byte[] data) where T : unmanaged
=> return Unsafe.As<byte, T>(ref data[0]);
.NET Framework/Standard Solution:
public static unsafe byte[] SerializeValueType<T>(in T value) where T : unmanaged
{
byte[] result = new byte[sizeof(T)];
fixed (byte* dst = result)
*(T*)dst = value;
return result;
}
// Note: Validation is omitted for simplicity
public static unsafe T DeserializeValueType<T>(byte[] data) where T : unmanaged
{
fixed (byte* src = data)
return *(T*)src;
}
See the complete code with validations here.
Remarks:
The OP's example contains a string, which is a reference type so the solution above cannot be used for that. And if you can't use generic methods for some reason things start to get more complicated, especially for .NET Framework (but non-generic size calculation is a pain also on the Core platform). If performance does not matter, then you can revert to Marshal.SizeOf and StructureToPtr as suggested by several other answers, or feel free to use the BinarySerializer.SerializeValueType method from my library that I linked also for the examples above (NuGet).
This can be done very straightforwardly.
Define your struct explicitly with [StructLayout(LayoutKind.Explicit)]
int size = list.GetLength(0);
IntPtr addr = Marshal.AllocHGlobal(size * sizeof(DataStruct));
DataStruct *ptrBuffer = (DataStruct*)addr;
foreach (DataStruct ds in list)
{
*ptrBuffer = ds;
ptrBuffer += 1;
}
This code can only be written in an unsafe context. You have to free addr when you're done with it.
Marshal.FreeHGlobal(addr);
I've come up with a different approach that could convert any struct without the hassle of fixing length, however the resulting byte array would have a little bit more overhead.
Here is a sample struct:
[StructLayout(LayoutKind.Sequential)]
public class HelloWorld
{
public MyEnum enumvalue;
public string reqtimestamp;
public string resptimestamp;
public string message;
public byte[] rawresp;
}
As you can see, all those structures would require adding the fixed length attributes. Which could often ended up taking up more space than required. Note that the LayoutKind.Sequential is required, as we want reflection to always gives us the same order when pulling for FieldInfo. My inspiration is from TLV Type-Length-Value. Let's have a look at the code:
public static byte[] StructToByteArray<T>(T obj)
{
using (MemoryStream ms = new MemoryStream())
{
FieldInfo[] infos = typeof(T).GetFields(BindingFlags.Public | BindingFlags.Instance);
foreach (FieldInfo info in infos)
{
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream inms = new MemoryStream()) {
bf.Serialize(inms, info.GetValue(obj));
byte[] ba = inms.ToArray();
// for length
ms.Write(BitConverter.GetBytes(ba.Length), 0, sizeof(int));
// for value
ms.Write(ba, 0, ba.Length);
}
}
return ms.ToArray();
}
}
The above function simply uses the BinaryFormatter to serialize the unknown size raw object, and I simply keep track of the size as well and store it inside the output MemoryStream too.
public static void ByteArrayToStruct<T>(byte[] data, out T output)
{
output = (T) Activator.CreateInstance(typeof(T), null);
using (MemoryStream ms = new MemoryStream(data))
{
byte[] ba = null;
FieldInfo[] infos = typeof(T).GetFields(BindingFlags.Public | BindingFlags.Instance);
foreach (FieldInfo info in infos)
{
// for length
ba = new byte[sizeof(int)];
ms.Read(ba, 0, sizeof(int));
// for value
int sz = BitConverter.ToInt32(ba, 0);
ba = new byte[sz];
ms.Read(ba, 0, sz);
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream inms = new MemoryStream(ba))
{
info.SetValue(output, bf.Deserialize(inms));
}
}
}
}
When we want to convert it back to its original struct we simply read the length back and directly dump it back into the BinaryFormatter which in turn dump it back into the struct.
These 2 functions are generic and should work with any struct, I've tested the above code in my C# project where I have a server and a client, connected and communicate via NamedPipeStream and I forward my struct as byte array from one and to another and converted it back.
I believe my approach might be better, since it doesn't fix length on the struct itself and the only overhead is just an int for every fields you have in your struct. There are also some tiny bit overhead inside the byte array generated by BinaryFormatter, but other than that, is not much.
I would take a look at the BinaryReader and BinaryWriter classes. I recently had to serialize data to a byte array (and back) and only found these classes after I'd basically rewritten them myself.
http://msdn.microsoft.com/en-us/library/system.io.binarywriter.aspx
There is a good example on that page too.
Looks like a predefined (C level) structure for some external library. Marshal is your friend. Check:
http://geekswithblogs.net/taylorrich/archive/2006/08/21/88665.aspx
for a starter how to deal with this. Note that you can - with attributes - define things like byte layout and string handling. VERY nice approach, actually.
Neither BinaryFormatter Nor MemoryStream are done for that.
#Abdel Olakara answer donese not work in .net 3.5, should be modified as below:
public static void ByteArrayToStructure<T>(byte[] bytearray, ref T obj)
{
int len = Marshal.SizeOf(obj);
IntPtr i = Marshal.AllocHGlobal(len);
Marshal.Copy(bytearray, 0, i, len);
obj = (T)Marshal.PtrToStructure(i, typeof(T));
Marshal.FreeHGlobal(i);
}
Header header = new Header();
Byte[] headerBytes = new Byte[Marshal.SizeOf(header)];
Marshal.Copy((IntPtr)(&header), headerBytes, 0, headerBytes.Length);
This should do the trick quickly, right?
This example here is only applicable to pure blittable types, e.g., types that can be memcpy'd directly in C.
Example - well known 64-bit struct
[StructLayout(LayoutKind.Sequential)]
public struct Voxel
{
public ushort m_id;
public byte m_red, m_green, m_blue, m_alpha, m_matid, m_custom;
}
Defined exactly like this, the struct will be automatically packed as 64-bit.
Now we can create volume of voxels:
Voxel[,,] voxels = new Voxel[16,16,16];
And save them all to a byte array:
int size = voxels.Length * 8; // Well known size: 64 bits
byte[] saved = new byte[size];
GCHandle h = GCHandle.Alloc(voxels, GCHandleType.Pinned);
Marshal.Copy(h.AddrOfPinnedObject(), saved, 0, size);
h.Free();
// now feel free to save 'saved' to a File / memory stream.
However, since the OP wants to know how to convert the struct itself, our Voxel struct can have following method ToBytes:
byte[] bytes = new byte[8]; // Well known size: 64 bits
GCHandle h = GCHandle.Alloc(this, GCHandleType.Pinned);
Marshal.Copy(hh.AddrOfPinnedObject(), bytes, 0, 8);
h.Free();

Categories