his question is about converting between a struct and a byte array. Many solutions are based around GCHandle.Alloc() and Marshal.StructureToPtr(). The problem is these calls generate garbage. For example, under Windows CE 6 R3 about 400 bytes of garbarge is made with a small structure. If the code below could be made to work the solution could be considered cleaner. It appears the sizeof() happens too late in the compile to work.
public struct Data
{
public double a;
public int b;
public double c;
}
[StructLayout(LayoutKind.Explicit)]
public unsafe struct DataWrapper
{
private static readonly int val = sizeof(Data);
[FieldOffset(0)]
public fixed byte Arr[val]; // "fixed" is to embed array instead of ref
[FieldOffset(0)]
public Data; // based on a C++ union
}
You can in fact use the constant value 1 as the size of the fixed array in DataWrapper. There is no bounds checking when accessing an unsafe fixed array nor is there any mechanism to request its length at runtime. The presence of the Data member at offset 0 is enough to ensure that you can actually read and write sizeof(Data) bytes to/from DataWrapper.Arr.
With that said, a more straight forward solution would be to cast a Data* pointer to byte* as follows:
unsafe {
Data d = new Data();
d.a = 125.5;
d.b = 0x7FEEDDEE;
d.c = 130.0;
byte* bytes = (byte*)&d;
for (int i = 0; i < sizeof(Data); i++) {
Console.WriteLine(i + ": " + bytes[i]);
}
}
Related
I have this structure in C++ that I need to be converted to C#, so I can create this structure from a byte[].
struct TRANS_RECORD_DATA {
int size, code;
int ThrowItemCount;
int ItemCount;
int ItemSubStart;
int DataSize;
BYTE Data[sizeof(sRECORD_ITEM) * 200]; // sizeof(sRECORD_ITEM) = 548
};
The C# version:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct TRANS_RECORD_DATA {
public int size, code;
public int ThrowItemCount;
public int ItemCount;
public int ItemSubStart;
public int DataSize;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 548*200)]
public byte[] Data;
};
I am using this generic function to give me the structure from the byte array:
T ByteArrayToStructure<T>(byte[] bytes) where T : struct
{
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
T stuff = (T) Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof (T));
handle.Free();
return stuff;
}
But it is giving me:
AccessViolation Exception.
I belive I know why but I don't know how to solve it. The byte[] I have, that needs to be mapped into the struct, does not always have the Data member with size of 548*200. This number is a maximum. But seems that that GenericMethod I use, always try to create the struct with that Data always 548*200, and then it will obviously throw an AccessViolation because the data to be mapped has ended already.
For example this code:
var bytes = new byte[26];
var structure = ByteArrayToStructure<TRANS_RECORD_DATA>(bytes);
Should return a TRANS_RECORD_DATA with all those int members with 0 value, and finally that byte[] Data would have only the remaining two bytes. (26 - 24 = 2). But it seems that it tries to create the full 548*200 byte[] all the time, and then cause the access violation.
Is there a way to solve this ?
So as you explaining it, regardless of C definition for Data array being 200*548 bytes long, it is not actually fully allocated or filled by an external unmanaged code you call.
this way your only workaround is NOT to define Data in your C# struct definition:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct TRANS_RECORD_DATA {
public int size, code;
public int ThrowItemCount;
public int ItemCount;
public int ItemSubStart;
public int DataSize;
};
and after reading DataSize re-interpret remainder of the byte array as that data structure.
you can still use series of calls to Marshal.PtrToStructure:
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
TRANS_RECORD_DATA stuff = (TRANS_RECORD_DATA) Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof (TRANS_RECORD_DATA));
var items = new List<sRECORD_ITEM>(stuff.ItemCount);
for (int i = 0; i < stuff.ItemCount; ++i)
{
var ptr = handle.AddrOfPinnedObject().Add(i*548)
sRECORD_ITEM item = (sRECORD_ITEM)Marshal.PtrToStructure(ptr,typeof(sRECORD_ITEM));
}
I am pretty sure this should do the trick.
I'm using an external C library that comes with a C# wrapper file containing a struct like (among many other things):
[StructLayout(LayoutKind.Sequential, Pack = 8)]
unsafe public struct Data
{
public fixed double Values[3];
};
I can get the values by adding an additional method to that struct:
public double[] CopyFixedDoubleArray()
{
double[] result = new double[3];
fixed (double* pDst = result)
{
fixed (double* pSrc = Values)
{
double* pd = pDst;
double* ps = pSrc;
for (int i = 0; i < result.Length; i++)
{
*pd = *ps;
pd++; ps++;
}
}
}
return result;
}
But this is not what I want because I don't want to touch the wrapper file (to avoid having to keep the external file in our SCM repository). Whatever I try to do to get the values from outside that struct results in the following error:
Fixed size buffers can only be accessed through locals or fields
What I've tried:
double a = data.Values[2];
double a; unsafe { a = data.Values[2]; }
double a; fixed (double* p = data.Values) { a = p[2]; }
fixed (double a = data.Values[2]) { /* use a */ }
Is there any way to get around this?
Suddenly the word 'extension method' crossed my mind. And yes, I have it working:
public static double[] CopyFixedDoubleArray(this Data data)
{
unsafe
{
return new[] { data.Values[0], data.Values[1], data.Values[2] };
}
}
Rather than patch your code up, have you considered shoving this over to the marshaller? That's its entire purpose: to assist with native interop while letting you write pure managed code.
You'd have to declare your struct like this:
[StructLayout(LayoutKind.Sequential, Pack = 8)]
public struct Data
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
public double[] Values;
};
When this structure is filled by the marshaller (through either a direct function call through a pointer, or from mapping the struct on top of a byte buffer) it will allocate the proper array and copy its contents over. This makes your entire function obsolete, but if you really wanted to you could simply use Array.Copy (or the Linq equivalent).
Notice there's no more unsafe modifier. You can run this in a partially trusted environment with no issues!
I have some struct like this
struct MyStruct
{
public int field1;
public int field2;
public int field3;
}
and I have pointer to array of this struct.
So, I need to get array from this pointer.
I'm tried to using Marshal.PtrToStructure, but i had memory reading error.
This is my methode:
public MyStruct[] GetArrayOfStruct(IntPtr pointerToStruct, int length)
{
var sizeInBytes = Marshal.SizeOf(typeof(TCnt));
MyStruct[] output = new MyStruct[length];
for (int i = 0; i < length; i++)
{
IntPtr p = new IntPtr((pointerToStruct.ToInt32() + i * sizeInBytes));
output[i] = (MyStruct)System.Runtime.InteropServices.Marshal.PtrToStructure(p, typeof(MyStruct));
}
return output;
}
So, what am i doing wrong ?
This function worked for me, assuming that the size of the struct is fixed:
public static void MarshalUnmananagedArray2Struct<T>(IntPtr unmanagedArray, int length, out T[] mangagedArray)
{
var size = Marshal.SizeOf(typeof(T));
mangagedArray = new T[length];
for (int i = 0; i < length; i++)
{
IntPtr ins = new IntPtr(unmanagedArray.ToInt64() + i * size);
mangagedArray[i] = Marshal.PtrToStructure<T>(ins);
}
}
Two problems. You use TCnt instead of MyStruct in the Marshal.SizeOf() call. Your IntPtr arithmetic cannot work on a 64-bit machine, you must use IntPtr.ToInt64() or cast to (long).
Just getting the wrong IntPtr or length is certainly a possibility too of course. Use Debug + Windows + Memory + Memory 1 and put "pointerToStruct" in the Address box for basic verification.
Structs in C and C# are not the same thing. One of the differences is that in C# you have to explicitly demand that your struct should be sequentially laid out. If you didn't write
[StructLayout(LayoutKind.Sequential)] or [StructLayout(LayoutKind.Explicit)] attribute to your structure I don't believe that you can manage it in this way. Microsoft states that PtrToStructure is to be used to convert structures from unmanaged to managed memory
You should test if adding this attributes to your struct helps, If it doesn't yet help try allocating memory with Marshal.AllocHGlobal(IntPtr) and use Marshal.Copy to init your structure and then try using PtrToStructure. If this works then you can't use PtrToStructure with managed memory
I'm following the pinvoke code provided here but am slightly scared by the marshalling of the variable-length array as size=1 and then stepping through it by calculating an offset instead of indexing into an array. Isn't there a better way? And if not, how should I do this to make it safe for 32-bit and 64-bit?
[StructLayout(LayoutKind.Sequential)]
public struct SID_AND_ATTRIBUTES
{
public IntPtr Sid;
public uint Attributes;
}
[StructLayout(LayoutKind.Sequential)]
public struct TOKEN_GROUPS
{
public int GroupCount;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
public SID_AND_ATTRIBUTES[] Groups;
};
public void SomeMethod()
{
IntPtr tokenInformation;
// ...
string retVal = string.Empty;
TOKEN_GROUPS groups = (TOKEN_GROUPS)Marshal.PtrToStructure(tokenInformation, typeof(TOKEN_GROUPS));
int sidAndAttrSize = Marshal.SizeOf(new SID_AND_ATTRIBUTES());
for (int i = 0; i < groups.GroupCount; i++)
{
// *** Scary line here:
SID_AND_ATTRIBUTES sidAndAttributes = (SID_AND_ATTRIBUTES)Marshal.PtrToStructure(
new IntPtr(tokenInformation.ToInt64() + i * sidAndAttrSize + IntPtr.Size),
typeof(SID_AND_ATTRIBUTES));
// ...
}
I see here another approach of declaring the length of the array as much bigger than it's likely to be, but that seemed to have its own problems.
As a side question: When I step through the above code in the debugger I'm not able to evaluate tokenInformation.ToInt64() or ToInt32(). I get an ArgumentOutOfRangeException. But the line of code executes just fine!? What's going on here?
Instead of guessing what the offset, is its generally better to use Marshal.OffsetOf(typeof(TOKEN_GROUPS), "Groups") to get the correct offset to the start of the array.
I think it looks okay -- as okay as any poking about in unmanaged land is, anyway.
However, I wonder why the start is tokenInformation.ToInt64() + IntPtr.Size and not tokenInformation.ToInt64() + 4 (as the GroupCount field type is an int and not IntPtr). Is this for packing/alignment of the structure or just something fishy? I do not know here.
Using tokenInformation.ToInt64() is important because on a 64-bit machine will explode (OverflowException) if the IntPtr value is larger than what an int can store. However, the CLR will handle a long just fine on both architectures and it doesn't change the actual value extracted from the IntPtr (and thus put back into the new IntPtr(...)).
Imagine this (untested) function as a convenience wrapper:
// unpacks an array of structures from unmanaged memory
// arr.Length is the number of items to unpack. don't overrun.
void PtrToStructureArray<T>(T[] arr, IntPtr start, int stride) {
long ptr = start.ToInt64();
for (int i = 0; i < arr.Length; i++, ptr += stride) {
arr[i] = (T)Marshal.PtrToStructure(new IntPtr(ptr), typeof(T));
}
}
var attributes = new SID_AND_ATTRIBUTES[groups.GroupCount];
PtrToStructureArray(attributes, new IntPtr(tokenInformation.ToInt64() + IntPtr.Size), sidAndAttrSize);
Happy coding.
C++ code:
struct tPacket
{
WORD word1;
WORD word2;
BYTE byte1;
BYTE byte2;
BYTE array123[8];
}
static char data[8192] = {0};
...
some code to fill up the array
...
tPacket * packet = (tPacket *)data;
We can't do that as easy in C#.
Please note there is an array in the C++ structure.
Alternatively, using this source file could do the job for us, but not if there is an array in the structure.
I'm unsure of exactly what you are asking. Are you trying to get an equivalent structure definition in C# for plain old C# usage or for interop (PInvoke) purposes? If it's for PInvoke the follownig structure will work
[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
public struct tPacket {
/// WORD->unsigned short
public ushort word1;
/// WORD->unsigned short
public ushort word2;
/// BYTE->unsigned char
public byte byte1;
/// BYTE->unsigned char
public byte byte2;
/// BYTE[8]
[System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst=8, ArraySubType=System.Runtime.InteropServices.UnmanagedType.I1)]
public byte[] array123;
}
If you are looking for a plain old C# structure that has the same characteristics, it's unfortunately not possible to do with a struct. You cannot define an inline array of a contstant size in a C# structure nor can you force the array to be a specific size through an initializer.
There are two alternative options in the managed world.
Use a struct which has a create method that fills out the array
[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
public struct tPacket {
public ushort word1;
public ushort word2;
public byte byte1;
public byte byte2;
public byte[] array123;
public static tPacket Create() {
return new tPacket() { array123 = new byte[8] };
}
}
Or alternatively use a class where you can initialize the array123 member variable directly.
EDIT OP watns to know how to convert a byte[] into a tPacket value
Unfortunately there is no great way to do this in C#. C++ was awesome for this kind of task because has a very weak type system in that you could choose to view a stream of bytes as a particular structure (evil pointer casting).
This may be possible in C# unsafe code but I do not believe it is.
Essentially what you will have to do is manually parse out the bytes and assign them to the various values in the struct. Or write a native method which does the C style casting and PInvoke into that function.
I think what you are looking for (if you are using a similar structure definition like JaredPar posted) is something like this:
tPacket t = new tPacket();
byte[] buffer = new byte[Marshal.SizeOf(typeof(tPacket))];
socket.Receive(buffer, 0, buffer.length, 0);
GCHandle pin = GCHandle.Alloc(buffer, GCHandleType.Pinned);
t = (tPacket)Marshal.PtrToStructure(pin.AddrOfPinnedObject(), typeof(tPacket));
pin.free();
//do stuff with your new tPacket t
It can be done with unsafe code too, although it restricts the context under which your program can run, and, naturally, introduces the possibility of security flaws.
The advantage is that you cast directly from an array to the structure using pointers and it's also maintenance-free if you are only going to add or remove fields from the struct. However, accessing the arrays require using the fixed-statement as the GC can still move the struct around in memory when it's contained in an object.
Here's some modified code of an unsafe struct I used for interpreting UDP packets:
using System;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Sequential)]
public unsafe struct UnsafePacket
{
int time;
short id0;
fixed float acc[3];
short id1;
fixed float mat[9];
public UnsafePacket(byte[] rawData)
{
if (rawData == null)
throw new ArgumentNullException("rawData");
if (sizeof(byte) * rawData.Length != sizeof(UnsafePacket))
throw new ArgumentException("rawData");
fixed (byte* ptr = &rawData[0])
{
this = *(UnsafePacket*)rawPtr;
}
}
public float GetAcc(int index)
{
if (index < 0 || index >= 3)
throw new ArgumentOutOfRangeException("index");
fixed (UnsafePacket* ptr = &acc)
{
return ptr[index];
}
}
public float GetMat(int index)
{
if (index < 0 || index >= 9)
throw new ArgumentOutOfRangeException("index");
fixed (UnsafePacket* ptr = &mat)
{
return ptr[index];
}
}
// etc. for other properties
}
For this kind of code it is extremely important to check that the length of the array perfectly matches the size of the struct, otherwise you'll open for some nasty buffer overflows. As the unsafe keyword has been applied to the whole struct, you don't need to mark each method or codeblock as separate unsafe statements.
You can place what looks to the outside world like an array of fixed size within a safe structure by writing functions within the structure for access. For example, here is a fixed 4 by 4 double precision array within a safe structure:
public struct matrix4 // 4 by 4 matrix
{
//
// Here we will create a square matrix that can be written to and read from similar
// (but not identical to) using an array. Reading and writing into this structure
// is slower than using an array (due to nested switch blocks, where nest depth
// is the dimensionality of the array, or 2 in this case). A big advantage of this
// structure is that it operates within a safe context.
//
private double a00; private double a01; private double a02; private double a03;
private double a10; private double a11; private double a12; private double a13;
private double a20; private double a21; private double a22; private double a23;
private double a30; private double a31; private double a32; private double a33;
//
public void AssignAllZeros() // Zero out the square matrix
{ /* code */}
public double Determinant() // Common linear algebra function
{ /* code */}
public double Maximum() // Returns maximum value in matrix
{ /* code */}
public double Minimum() // Minimum value in matrix
{ /* code */}
public double Read(short row, short col) // Outside read access
{ /* code */}
public double Read(int row, int col) // Outside read access overload
{ /* code */}
public double Sum() // Sum of 16 double precision values
{
return a00 + a01 + a02 + a03 + a10 + a11 + a12 + a13 + a20 + a21 + a22 + a23 + a30 + a31 + a32 + a33;
}
public void Write(short row, short col, double doubleValue) // Write access to matrix
{ /* code */}
public void Write(int row, int col, double doubleValue) // Write access overload
{ /* code */}
}