I have this method to convert long to little endian byte array
public static byte[] UnsignedIntegerToLEByteArray(ulong value)
{
// Value in bytes... in your system's endianness (let's say: little endian)
byte[] bytes = BitConverter.GetBytes(value);
// If it was big endian, reverse it
if (!BitConverter.IsLittleEndian)
Array.Reverse(bytes);
return bytes;
}
My goal is to use it for shorted data types too, like int, short, etc. See here:
byte a = 0xAA;
ushort b = 0xEEAA;
uint c = 0xAABBCCDD;
ulong d = 0xAABBCCDDAAAAAABB;
// If you passed short below, you are only interested
// in first two bytes of the array
byte []tmp = DppUtilities.UnsignedIntegerToLEByteArray(b);
This works if my machine is little endian.
Will it also work if it is run on big endian machine? I think yes, but I would like to verify.
You can play a trick with a help of IntPtr and Marshal in order to convert any struct (including byte, ushort and ulong):
// Disclaimer: the structure will be reversed as a whole, not field by field
public static byte[] ToLEByteArray<T>(T value) where T: struct {
int size = Marshal.SizeOf(typeof(T));
byte[] bytes = new byte[size];
IntPtr p = Marshal.AllocHGlobal(size);
try {
Marshal.StructureToPtr(value, p, true);
Marshal.Copy(p, bytes, 0, size);
}
finally {
Marshal.FreeHGlobal(p);
}
// If it was big endian, reverse it
if (!BitConverter.IsLittleEndian)
Array.Reverse(bytes);
return bytes;
}
....
Byte b = 123;
ushort s = 123;
ulong l = 123;
Byte[] result_byte = ToLEByteArray(b);
Byte[] result_ushort = ToLEByteArray(s);
Byte[] result_ulong = ToLEByteArray(l);
....
int i = 123456;
Byte[] result_int = ToLEByteArray(i);
EDIT: what's wrong with the implementation in the question? (from the comment).
Or, restating the question, what is that stuff with IntPtr, Marshal for?
The main issue of the question's implementation is the initial conversion to ulong:
// all the arguments will be converted to ulong
public static byte[] UnsignedIntegerToLEByteArray(ulong value)
In order to illustrate the problem, imagine, that we have two values
Byte x = 0x12; // 18
ulong u = 0x0000000000000012; // 18
and the expected output is
new byte[] {0x12}; // for a single byte
new byte[] {0x12, 0, 0, 0, 0, 0, 0, 0}; // for 8 bytes, i.e. ulong
however, the actual output will be
new byte[] {0x12, 0, 0, 0, 0, 0, 0, 0};
for both byte and ulong. This misbehaviour can lead to problems if you, say, want to write down numeric values (byte, short, ulong etc.), to a binary file, pass them to a binary stream etc:
using (Stream stm = ...) {
...
Byte[] buffer = UnsignedIntegerToLEByteArray(...);
stm.Write(buffer, offset, buffer.Length); // <- the (possibly!) erroneous write
...
}
I had the same thing and I tested it.
I can tell you it works 100%. If you want to check it out yourself, you can for example just pretend you need it the other way. So you reverse the byte array and reverse it if it's IsLittleEndian.
Your output is always in LittleEndian in your method, which is what you want.
Like BitConverter.GetBytes has different overloads for that, you should do it the same way:
public static byte[] UnsignedIntegerToLEByteArray(ulong value) { ...
public static byte[] UnsignedIntegerToLEByteArray(int value) { ...
public static byte[] UnsignedIntegerToLEByteArray(short value) { ...
The complier chooses the right overload when using it like you already did:
byte []tmp = DppUtilities.UnsignedIntegerToLEByteArray(b);
If you try to do this with a single method that try to check the zero bytes, it will produce wrong results:
How to tell if the number 00 00 12 34 should reverse 2 or 4 bytes?
Related
I have a structure which I want to send to a TCP client throught TCP protocol so I want to assign or copy this struct data to byte array:
struct StartReadXML
{
public int CmdID;//3
public char[] CmdName;//ReadXML
public char[] Description;//Other data
};
here am assigning data to struct data members as below :
StartReadXML startXML=new StartReadXML();
startXML.CmdID = 3;
startXML.CmdName = "sreedhar".ToCharArray();
startXML.Description = "Kumar".ToCharArray();
Now, I want it to be assigned to a byte array. Which am doing using marshalling as below:
int sizestartXML = Marshal.SizeOf(startXML);//Get size of struct data
byte[] startXML_buf = new byte[sizestartXML];//byte array & its size
IntPtr ptr = Marshal.AllocHGlobal(sizestartXML);//pointer to byte array
Marshal.StructureToPtr(startXML, ptr, true);
Marshal.Copy(ptr, startXML_buf, 0, sizestartXML);
Marshal.FreeHGlobal(ptr);
//Sending struct data packet
stm.Write(startXML_buf, 0, startXML_buf.Length);//Modified
But, it fails at Structuretoptr conversion method. Please help in transferring the struct data as bytes for which am using above steps.
Thanks in advance Smile | :) !!
You cannot call StructureToPtr on arrays of variable size.
What this boils down to is that unless you know the size of CmdName and declare it - if it would be for example, 20 chars in size, like so:
public fixed char[] CmdName[20];
You will be greeted with an exception from the Marshal saying that your structure is either non-blittable or no meaningful size can be obtained.
This is a requirement the CLR imposes, and you can not work around.
An alternative method would be to use the Convert class or a serializer to convert the members of your struct manually, but unless you know the size of those arrays up front, you won't be able to use StructureToPtr - the same goes for the string type, as I'm assuming that's what your char array will contain.
Consider using a MemoryStream and writing values to the stream, and sending the contents of the stream using stream.ToArray() instead.
Considering that you can't simply convert to binary a struct, use for example:
class StartReadXML
{
public int CmdID;//3
public string CmdName;//ReadXML
public string Description;//Other data
}
Then:
var srx = new StartReadXML();
srx.CmdID = 3;
srx.CmdName = "sreedhar";
srx.Description = "Kumar";
// Example of how to Write to byte[] buffer
byte[] buffer;
using (var ms = new MemoryStream())
{
using (var bw = new BinaryWriter(ms, Encoding.UTF8))
{
bw.Write(srx.CmdID);
bw.Write(srx.CmdName);
bw.Write(srx.Description);
}
buffer = ms.ToArray();
}
// Example of how to Read from byte[] buffer
var srx2 = new StartReadXML();
using (var ms = new MemoryStream(buffer))
{
using (var br = new BinaryReader(ms, Encoding.UTF8))
{
srx2.CmdID = br.ReadInt32();
srx2.CmdName = br.ReadString();
srx2.Description = br.ReadString();
}
}
Note that here I'm working with a "variable length" packet: the length of CmdName and Description aren't fixed (and BinaryWriter/BinaryReader handle this by prepending the length of the string).
The opposite is when you have a packet of fixed length, with fixed-length strings. That requires a totally different handling.
I am writing my custom serializer for my class which should serialize a class object to a byte array. Below is my sample code:
[StructLayout(LayoutKind.Sequential)]
public class ATTRIBUTE_STR
{
public uint atType;
public uint atLen;
public byte[] atData = new byte[3];
public byte[] serialize()
{
byte[] temp = new byte[Marshal.SizeOf(this)];
byte[] buffer = BitConverter.GetBytes(atType);
Array.Copy(buffer, 0, temp, 0, buffer.Length);
buffer = null;
buffer = BitConverter.GetBytes(atLen);
Array.Copy(buffer, 0, temp, 4, buffer.Length);
Array.Copy(this.atData, 0, temp, 8, this.atData.Length);
return temp;
}
};
However, the byte array atData because of how its stored in memory with byte alignment and all, it's not getting correctly into the temp byte array. Since the byte arrays have even byte alignment.
How can I serialize this object accounting for the alignment of members in memory?
EDIT: I know there are other options like Marshalling or ProtocolBuffers, but I would like using my custom serializer.
Your sizeof call is probably not returning what you think it is, but there hardly seems much point worrying about Sizeof if you're going to stick with hard-coded offsets for the output.]
So you might as well do something like this:
byte[] temp = new byte[8+atLen.Length];
BitConverter.GetBytes(atType).CopyTo(temp,0);
BitConverter.GetBytes(atLen).CopyTo(temp,4);
this.AtData.CopyTo(temp,8);
return temp;
Update: Sizeof returns the amount of unmanaged space necessary to store your object - but bear in mind that your atData array contents are not stored within your object, they are in an array object which is referred to by your object. So Marshal.Sizeof is not going to give you an answer which includes the size of your array - indeed, if you ask Marshal.Sizeof to give you Sizeof(atData), you'll get an exception.
Some code I'm modifying makes extensive use of structs to communicate with some factory equipment by loading them to or from byte arrays.
Here's a made-up ultra-simplified example of such a struct.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct K_FOO
{
public byte a_byte; // 1 byte
public BYTE3_TYPE b3; // 3 bytes
public int num; // happens to be 4 bytes
}
BYTE3_TYPE looks like this...
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class BYTE3_TYPE
{
[System.Runtime.InteropServices.MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
public byte[] L = new byte[3];
}
If I just do a
K_FOO origFoo = new K_FOO();
the int and the byte are initialized to 0's, I assume because they are native types, but the byte array b3 is *un*initialized - all nulls. I have to explicitly load it, for example,
BYTE3_TYPE b3 = new BYTE3_TYPE();
origFoo.b3 = b3;
... and I couldn't think of any alternative because structs don't take parameterless constructors but the real structs are huge.
But I noticed something interesting. We have routines to copy these structs to and from byte arrays. For example . . .
public static T ByteArrayToStructure<T>(byte[] buffer) where T : struct
{
int length = buffer.Length;
IntPtr ptr = Marshal.AllocHGlobal(length); // allocate (length) bytes
Marshal.Copy(buffer, 0, ptr, length); // copies into UNmanaged space
T result = (T)Marshal.PtrToStructure(ptr, typeof(T));
Marshal.FreeHGlobal(ptr);
return result;
}
...and if I call it...
K_FOO retFoo = ByteArrayToStructure<K_FOO>(buffer.bytes);
... the resulting struct is returned fully initialized, the bytes in the byte array have all had their space allocated so they could be loaded, apparently in the PtrToStructure() call. This implies .Net "knows" how to initialize such a struct. So is there some way to get .Net to do that for me so I can avoid writing hundreds of lines of explicit initialization code? Thanks in advance!
If you make your BYTE3_TYPE a struct instead of a class, the default constructor (calling K_FOO origFoo = new K_FOO();) will initialize the entire thing to zero correctly.
This is also likely the correct approach if you're trying to match existing specifications and pass this to custom hardware.
How can I convert sbyte[] to base64 string?
I cannot convert that sbyte[] to a byte[], to keep interoperability with java.
You absolutely can convert the sbyte[] to a byte[] - I can pretty much guarantee you that the Java code will really be treating the byte array as unsigned. (Or to put it another way: base64 is only defined in terms of unsigned bytes...)
Just convert to byte[] and call Convert.ToBase64String. Converting to byte[] is actually really easy - although C# itself doesn't provide a conversion between the two, the CLR is quite happy to perform a reference conversion, so you just need to fool the C# compiler:
sbyte[] x = { -1, 1 };
byte[] y = (byte[]) (object) x;
Console.WriteLine(Convert.ToBase64String(y));
If you want to have a genuine byte[] you can copy:
byte[] y = new byte[x.Length];
Buffer.BlockCopy(x, 0, y, 0, y.Length);
but personally I'd stick with the first form.
class Program
{
static void Main()
{
sbyte[] signedByteArray = { -2, -1, 0, 1, 2 };
byte[] unsignedByteArray = (byte[])(Array)signedByteArray;
Console.WriteLine(Convert.ToBase64String(unsignedByteArray));
}
}
I have a TCP socket connection where I get byte array of data which I have to decrypt and then format, so I am looking for a simple way to read this resulted byte array that I can easyly walk thru the elements exporting it as I need to a type or variable etc, to explain it better see the below example:
Let's say we have the byte array (packets are little-endian, this is just hypotetical example):
01 02 00 03 74 00 74 00 69 00
I want to put the first 2 bytes into an int, the next 2 bytes into another int and rest as an string.
So it would look like this translated:
int first = 258;
int seconnd = 3;
string rest = "tti"
I am not a java person, but on java there was something interesting I noticed somewhere of people reading packets directly like this:
messageSize = readH(); // for 2 bytes
key = readH(); // for 2 bytes
message = readS(); // read as string
And I was wondering if there is similar pre-made feature in c# or any other resource that may help me better doing this task ?
You can use BinaryReader like this:
using (var reader = new BinaryReader(stream))
{
var first = reader.ReadUInt16();
var second = reader.ReadUInt16();
var stringBytes = reader.ReadBytes(6);
var str = Encoding.Unicode.GetString(stringBytes);
}
However, this will only work if little-endian is used.
The example you posted is big-endian.
I assume that you implement both writer and sender in C#, so you are good to go with BinaryReader and BinaryWriter, they both use little-endian, so they will understand each other.
[Edit]
Another approach you might want to consider is using struct.
For example in your case :
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct MyStruct
{
public ushort First;
public ushort Second;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
public string MyString;
}
The code would look like this
var myStruct = new MyStruct { First = 1, Second = 2, MyString = "asd" };
var bytes = StructToBytes(myStruct);
var myStruct1 = BytesToStruct<MyStruct>(bytes);
And two utility methods:
public static T BytesToStruct<T>(byte[] bytes) where T : struct
{
AssertUtilities.ArgumentNotNull(bytes, "bytes");
var structSize = Marshal.SizeOf(typeof(T));
var pointer = IntPtr.Zero;
try
{
pointer = Marshal.AllocHGlobal(structSize);
Marshal.Copy(bytes, 0, pointer, structSize);
return (T)Marshal.PtrToStructure(pointer, typeof(T));
}
finally
{
if (pointer != IntPtr.Zero)
Marshal.FreeHGlobal(pointer);
}
}
public static byte[] StructToBytes<T>(T structObject) where T : struct
{
var structSize = Marshal.SizeOf(typeof(T));
var bytes = new byte[structSize];
var pointer = IntPtr.Zero;
try
{
pointer = Marshal.AllocHGlobal(structSize);
Marshal.StructureToPtr(structObject, pointer, true);
Marshal.Copy(pointer, bytes, 0, structSize);
return bytes;
}
finally
{
if (pointer != IntPtr.Zero)
Marshal.FreeHGlobal(pointer);
}
}
For simple examples, you could do something like:
static class StreamHelpers
{
public static int ReadH(this Stream stream)
{
int x = stream.ReadByte(), y = stream.ReadByte();
if (x < 0 || y < 0) throw new EndOfStreamException();
return (y << 8) | x;
}
...
}
which gives you access to:
int i = stream.ReadH();
etc. However, personally I would be writing a custom blahReader (for your blah), with an internal byte[] buffer and tracking cursor. This allows must more efficient reading in most cases (even compared to a BufferedStream).
As a side-note, your string looks to be UTF-16 and read to the EOF; reading to EOF is a pain as it doesn't really allow you to encode 2 strings. I would (instead) be adding (first) the length of the string, and I would be using UTF-8 unless there is a good chance of large character points. Then reading a string is a case of:
read the length
decode that much via Encoding.*
As a final point - if you are mostly writing small integers, there are alternative encoding styles for integers that might be useful (allowing single-byte for small values, but still allowing full int ranges to be expressed).