We Keep Coding

correct marshalling of c++ structure in c#

I have this structure in C++ that I need to be converted to C#, so I can create this structure from a byte[].
struct TRANS_RECORD_DATA {
int size, code;
int ThrowItemCount;
int ItemCount;
int ItemSubStart;
int DataSize;
BYTE Data[sizeof(sRECORD_ITEM) * 200]; // sizeof(sRECORD_ITEM) = 548
};
The C# version:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct TRANS_RECORD_DATA {
public int size, code;
public int ThrowItemCount;
public int ItemCount;
public int ItemSubStart;
public int DataSize;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 548*200)]
public byte[] Data;
};
I am using this generic function to give me the structure from the byte array:
T ByteArrayToStructure<T>(byte[] bytes) where T : struct
{
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
T stuff = (T) Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof (T));
handle.Free();
return stuff;
}
But it is giving me:
AccessViolation Exception.
I belive I know why but I don't know how to solve it. The byte[] I have, that needs to be mapped into the struct, does not always have the Data member with size of 548*200. This number is a maximum. But seems that that GenericMethod I use, always try to create the struct with that Data always 548*200, and then it will obviously throw an AccessViolation because the data to be mapped has ended already.
For example this code:
var bytes = new byte[26];
var structure = ByteArrayToStructure<TRANS_RECORD_DATA>(bytes);
Should return a TRANS_RECORD_DATA with all those int members with 0 value, and finally that byte[] Data would have only the remaining two bytes. (26 - 24 = 2). But it seems that it tries to create the full 548*200 byte[] all the time, and then cause the access violation.
Is there a way to solve this ?

So as you explaining it, regardless of C definition for Data array being 200*548 bytes long, it is not actually fully allocated or filled by an external unmanaged code you call.
this way your only workaround is NOT to define Data in your C# struct definition:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct TRANS_RECORD_DATA {
public int size, code;
public int ThrowItemCount;
public int ItemCount;
public int ItemSubStart;
public int DataSize;
};
and after reading DataSize re-interpret remainder of the byte array as that data structure.
you can still use series of calls to Marshal.PtrToStructure:
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
TRANS_RECORD_DATA stuff = (TRANS_RECORD_DATA) Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof (TRANS_RECORD_DATA));
var items = new List<sRECORD_ITEM>(stuff.ItemCount);
for (int i = 0; i < stuff.ItemCount; ++i)
{
var ptr = handle.AddrOfPinnedObject().Add(i*548)
sRECORD_ITEM item = (sRECORD_ITEM)Marshal.PtrToStructure(ptr,typeof(sRECORD_ITEM));
}
I am pretty sure this should do the trick.

How to convert C# Struct to Byte Array

I have a structure which I want to send to a TCP client throught TCP protocol so I want to assign or copy this struct data to byte array:
struct StartReadXML
{
public int CmdID;//3
public char[] CmdName;//ReadXML
public char[] Description;//Other data
};
here am assigning data to struct data members as below :
StartReadXML startXML=new StartReadXML();
startXML.CmdID = 3;
startXML.CmdName = "sreedhar".ToCharArray();
startXML.Description = "Kumar".ToCharArray();
Now, I want it to be assigned to a byte array. Which am doing using marshalling as below:
int sizestartXML = Marshal.SizeOf(startXML);//Get size of struct data
byte[] startXML_buf = new byte[sizestartXML];//byte array & its size
IntPtr ptr = Marshal.AllocHGlobal(sizestartXML);//pointer to byte array
Marshal.StructureToPtr(startXML, ptr, true);
Marshal.Copy(ptr, startXML_buf, 0, sizestartXML);
Marshal.FreeHGlobal(ptr);
//Sending struct data packet
stm.Write(startXML_buf, 0, startXML_buf.Length);//Modified
But, it fails at Structuretoptr conversion method. Please help in transferring the struct data as bytes for which am using above steps.
Thanks in advance Smile | :) !!

You cannot call StructureToPtr on arrays of variable size.
What this boils down to is that unless you know the size of CmdName and declare it - if it would be for example, 20 chars in size, like so:
public fixed char[] CmdName[20];
You will be greeted with an exception from the Marshal saying that your structure is either non-blittable or no meaningful size can be obtained.
This is a requirement the CLR imposes, and you can not work around.
An alternative method would be to use the Convert class or a serializer to convert the members of your struct manually, but unless you know the size of those arrays up front, you won't be able to use StructureToPtr - the same goes for the string type, as I'm assuming that's what your char array will contain.
Consider using a MemoryStream and writing values to the stream, and sending the contents of the stream using stream.ToArray() instead.

Considering that you can't simply convert to binary a struct, use for example:
class StartReadXML
{
public int CmdID;//3
public string CmdName;//ReadXML
public string Description;//Other data
}
Then:
var srx = new StartReadXML();
srx.CmdID = 3;
srx.CmdName = "sreedhar";
srx.Description = "Kumar";
// Example of how to Write to byte[] buffer
byte[] buffer;
using (var ms = new MemoryStream())
{
using (var bw = new BinaryWriter(ms, Encoding.UTF8))
{
bw.Write(srx.CmdID);
bw.Write(srx.CmdName);
bw.Write(srx.Description);
}
buffer = ms.ToArray();
}
// Example of how to Read from byte[] buffer
var srx2 = new StartReadXML();
using (var ms = new MemoryStream(buffer))
{
using (var br = new BinaryReader(ms, Encoding.UTF8))
{
srx2.CmdID = br.ReadInt32();
srx2.CmdName = br.ReadString();
srx2.Description = br.ReadString();
}
}
Note that here I'm working with a "variable length" packet: the length of CmdName and Description aren't fixed (and BinaryWriter/BinaryReader handle this by prepending the length of the string).
The opposite is when you have a packet of fixed length, with fixed-length strings. That requires a totally different handling.

Initialization of a struct in C#?

Some code I'm modifying makes extensive use of structs to communicate with some factory equipment by loading them to or from byte arrays.
Here's a made-up ultra-simplified example of such a struct.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct K_FOO
{
public byte a_byte; // 1 byte
public BYTE3_TYPE b3; // 3 bytes
public int num; // happens to be 4 bytes
}
BYTE3_TYPE looks like this...
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class BYTE3_TYPE
{
[System.Runtime.InteropServices.MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
public byte[] L = new byte[3];
}
If I just do a
K_FOO origFoo = new K_FOO();
the int and the byte are initialized to 0's, I assume because they are native types, but the byte array b3 is *un*initialized - all nulls. I have to explicitly load it, for example,
BYTE3_TYPE b3 = new BYTE3_TYPE();
origFoo.b3 = b3;
... and I couldn't think of any alternative because structs don't take parameterless constructors but the real structs are huge.
But I noticed something interesting. We have routines to copy these structs to and from byte arrays. For example . . .
public static T ByteArrayToStructure<T>(byte[] buffer) where T : struct
{
int length = buffer.Length;
IntPtr ptr = Marshal.AllocHGlobal(length); // allocate (length) bytes
Marshal.Copy(buffer, 0, ptr, length); // copies into UNmanaged space
T result = (T)Marshal.PtrToStructure(ptr, typeof(T));
Marshal.FreeHGlobal(ptr);
return result;
}
...and if I call it...
K_FOO retFoo = ByteArrayToStructure<K_FOO>(buffer.bytes);
... the resulting struct is returned fully initialized, the bytes in the byte array have all had their space allocated so they could be loaded, apparently in the PtrToStructure() call. This implies .Net "knows" how to initialize such a struct. So is there some way to get .Net to do that for me so I can avoid writing hundreds of lines of explicit initialization code? Thanks in advance!

If you make your BYTE3_TYPE a struct instead of a class, the default constructor (calling K_FOO origFoo = new K_FOO();) will initialize the entire thing to zero correctly.
This is also likely the correct approach if you're trying to match existing specifications and pass this to custom hardware.

Marshalled C string missing character

I am having a problem with marshalling a C character array. I have the following C# structure:
[StructLayout(LayoutKind.Explicit, Size = 16, CharSet = CharSet.Ansi), Serializable]
internal struct Header
{
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 4)]
[FieldOffset(0)]
public string header;
[FieldOffset(4)]
public int version;
[FieldOffset(8)]
public int diroffset;
[FieldOffset(12)]
public int direntries;
}
and the following code to read this structure from a stream:
public static T ReadStruct<T>(this Stream stream) where T : struct
{
var sz = Marshal.SizeOf(typeof(T));
var buffer = new byte[sz];
stream.Read(buffer, 0, sz);
var pinnedBuffer = GCHandle.Alloc(buffer, GCHandleType.Pinned);
var structure = (T) Marshal.PtrToStructure(
pinnedBuffer.AddrOfPinnedObject(), typeof(T));
pinnedBuffer.Free();
return structure;
}
Now my problem is that the header field misses a character after the struct is read. The file where the struct is read from contains the four bytes VPVP but after the struct has been read by ReadStruct the header string only contains VPV. If I take a look at the byte array in the read function in the debugger then that array contains the values 86, 80, 86, 80 which is VPVP. I also tried using LayoutKind.Sequential for the StructLayout but that didn't change anything.
Am I doing something wrong or why is there a character missing in my string?

The problem you're having lies in the struct definition, not in writing the bytes to it.
The problem lies right here:
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 4)]
As you've stated, you're writing out the text VPVP, which is 4 characters long, you'd think. This, however, is not the case. In C, you could declare the string as such:
char mystring[] = { 'V', 'P', 'V', 'P', '\0' };
You need that null character (\0) at the end, to mark off the end of the string. You need to take this into account when marshalling, because you need to reserve space for that "null terminator byte", if you do not, the C# string will add it for you in the available memory, so it will eat away your last character. So if you're gonna use a null-terminated string, you will have to make it of length 5.
EDIT: Here is a better solution, where you don't have to worry about null-terminators, you just use a char[] (and you also keep the magic 16 byte size):
[StructLayout(LayoutKind.Explicit, Size = 16, CharSet = CharSet.Ansi), Serializable]
internal struct Header
{
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 4)]
[FieldOffset(0)]
private char[] headerCharArray;
public string header
{
get { return new string(headerCharArray); }
set
{
if (value.Length == 4)
{
headerCharArray = value.ToArray();
}
else
{
throw new InvalidOperationException("String length was not 4.");
}
}
}
[FieldOffset(4)]
public int version;
[FieldOffset(8)]
public int diroffset;
[FieldOffset(12)]
public int direntries;
}
That way the char[] is stored in memory, and you can access it as a string through the property, which doesn't take in any memory of the struct itself.

Reading byte array to type/variable in a simplified way?

I have a TCP socket connection where I get byte array of data which I have to decrypt and then format, so I am looking for a simple way to read this resulted byte array that I can easyly walk thru the elements exporting it as I need to a type or variable etc, to explain it better see the below example:
Let's say we have the byte array (packets are little-endian, this is just hypotetical example):
01 02 00 03 74 00 74 00 69 00
I want to put the first 2 bytes into an int, the next 2 bytes into another int and rest as an string.
So it would look like this translated:
int first = 258;
int seconnd = 3;
string rest = "tti"
I am not a java person, but on java there was something interesting I noticed somewhere of people reading packets directly like this:
messageSize = readH(); // for 2 bytes
key = readH(); // for 2 bytes
message = readS(); // read as string
And I was wondering if there is similar pre-made feature in c# or any other resource that may help me better doing this task ?

You can use BinaryReader like this:
using (var reader = new BinaryReader(stream))
{
var first = reader.ReadUInt16();
var second = reader.ReadUInt16();
var stringBytes = reader.ReadBytes(6);
var str = Encoding.Unicode.GetString(stringBytes);
}
However, this will only work if little-endian is used.
The example you posted is big-endian.
I assume that you implement both writer and sender in C#, so you are good to go with BinaryReader and BinaryWriter, they both use little-endian, so they will understand each other.
[Edit]
Another approach you might want to consider is using struct.
For example in your case :
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct MyStruct
{
public ushort First;
public ushort Second;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
public string MyString;
}
The code would look like this
var myStruct = new MyStruct { First = 1, Second = 2, MyString = "asd" };
var bytes = StructToBytes(myStruct);
var myStruct1 = BytesToStruct<MyStruct>(bytes);
And two utility methods:
public static T BytesToStruct<T>(byte[] bytes) where T : struct
{
AssertUtilities.ArgumentNotNull(bytes, "bytes");
var structSize = Marshal.SizeOf(typeof(T));
var pointer = IntPtr.Zero;
try
{
pointer = Marshal.AllocHGlobal(structSize);
Marshal.Copy(bytes, 0, pointer, structSize);
return (T)Marshal.PtrToStructure(pointer, typeof(T));
}
finally
{
if (pointer != IntPtr.Zero)
Marshal.FreeHGlobal(pointer);
}
}
public static byte[] StructToBytes<T>(T structObject) where T : struct
{
var structSize = Marshal.SizeOf(typeof(T));
var bytes = new byte[structSize];
var pointer = IntPtr.Zero;
try
{
pointer = Marshal.AllocHGlobal(structSize);
Marshal.StructureToPtr(structObject, pointer, true);
Marshal.Copy(pointer, bytes, 0, structSize);
return bytes;
}
finally
{
if (pointer != IntPtr.Zero)
Marshal.FreeHGlobal(pointer);
}
}

For simple examples, you could do something like:
static class StreamHelpers
{
public static int ReadH(this Stream stream)
{
int x = stream.ReadByte(), y = stream.ReadByte();
if (x < 0 || y < 0) throw new EndOfStreamException();
return (y << 8) | x;
}
...
}
which gives you access to:
int i = stream.ReadH();
etc. However, personally I would be writing a custom blahReader (for your blah), with an internal byte[] buffer and tracking cursor. This allows must more efficient reading in most cases (even compared to a BufferedStream).
As a side-note, your string looks to be UTF-16 and read to the EOF; reading to EOF is a pain as it doesn't really allow you to encode 2 strings. I would (instead) be adding (first) the length of the string, and I would be using UTF-8 unless there is a good chance of large character points. Then reading a string is a case of:
read the length
decode that much via Encoding.*
As a final point - if you are mostly writing small integers, there are alternative encoding styles for integers that might be useful (allowing single-byte for small values, but still allowing full int ranges to be expressed).