We Keep Coding

create byte array from set of integers

Given these integers:
public uint ServerSequenceNumber;
public uint Reserved1;
public uint Reserved2;
public byte Reserved3;
public byte TotalPlayers;
What's the best way to create a byte[] array from them? If all their values are 1 the resulting array would be:
00000000000000000000000000000001 00000000000000000000000000000001 00000000000000000000000000000001 00000001 00000001

This should do what your looking for. BitConverter returns a byte array in the order of endianness of the processor being used. For x86 processors it is little-endian. This places the least significant byte first.
int value;
byte[] byte = BitConverter.GetBytes(value);
Array.Reverse(byte);
byte[] result = byte;
If you don't know the processor your going to be using the app on I suggest using:
int value;
byte[] bytes = BitConverter.GetBytes(value);
if (BitConverter.IsLittleEndian){
Array.Reverse(bytes);
}
byte[] result = bytes;

How's this?
byte[] bytes = new byte[14];
int i = 0;
foreach(uint num in new uint[]{SecureSequenceNumber, Reserved1, Reserved2})
{
bytes[i] = (byte)(num >> 24);
bytes[i + 1] = (byte)(num >> 16);
bytes[i + 2] = (byte)(num >> 8);
bytes[i + 3] = (byte)num;
i += 4;
}
bytes[12] = Reserved3;
bytes[13] = TotalPlayers;

Expanding on #Robert's answer I created a simple class that makes things neater when you're doing lots of concatanations:
class ByteJoiner
{
private int i;
public byte[] Bytes { get; private set; }
public ByteJoiner(int totalBytes)
{
i = 0;
Bytes = new byte[totalBytes];
}
public void Add(byte input)
{
Add(BitConverter.GetBytes(input));
}
public void Add(uint input)
{
Add(BitConverter.GetBytes(input));
}
public void Add(ushort input)
{
Add(BitConverter.GetBytes(input));
}
public void Add(byte[] input)
{
System.Buffer.BlockCopy(input, 0, Bytes, i, input.Length);
i += input.Length;
}
}

How to get little endian data from big endian in c# using bitConverter.ToInt32 method?

I am making application in C# which has a byte array containing hex values.
I am getting data as a big-endian but I want it as a little-endian and I am using Bitconverter.toInt32 method for converting that value to integer.
My problem is that before converting the value, I have to copy that 4 byte data into temporary array from source byte array and then reverse that temporary byte array.
I can't reverse source array because it also contains other data.
Because of that my application becomes slow.
In the code I have one source array of byte as waveData[] which contains a lot of data.
byte[] tempForTimestamp=new byte[4];
tempForTimestamp[0] = waveData[290];
tempForTimestamp[1] = waveData[289];
tempForTimestamp[2] = waveData[288];
tempForTimestamp[3] = waveData[287];
int number = BitConverter.ToInt32(tempForTimestamp, 0);
Is there any other method for that conversion?

Add a reference to System.Memory nuget and use BinaryPrimitives.ReverseEndianness().
using System.Buffers.Binary;
number = BinaryPrimitives.ReverseEndianness(number);
It supports both signed and unsigned integers (byte/short/int/long).

In modern-day Linq the one-liner and easiest to understand version would be:
int number = BitConverter.ToInt32(waveData.Skip(286).Take(4).Reverse().ToArray(), 0);
You could also...
byte[] tempForTimestamp = new byte[4];
Array.Copy(waveData, 287, tempForTimestamp, 0, 4);
Array.Reverse(tempForTimestamp);
int number = BitConverter.ToInt32(tempForTimestamp);
:)

If you know the data is big-endian, perhaps just do it manually:
int value = (buffer[i++] << 24) | (buffer[i++] << 16)
| (buffer[i++] << 8) | buffer[i++];
this will work reliably on any CPU, too. Note i is your current offset into the buffer.
Another approach would be to shuffle the array:
byte tmp = buffer[i+3];
buffer[i+3] = buffer[i];
buffer[i] = tmp;
tmp = buffer[i+2];
buffer[i+2] = buffer[i+1];
buffer[i+1] = tmp;
int value = BitConverter.ToInt32(buffer, i);
i += 4;
I find the first immensely more readable, and there are no branches / complex code, so it should work pretty fast too. The second could also run into problems on some platforms (where the CPU is already running big-endian).

Here you go
public static int SwapEndianness(int value)
{
var b1 = (value >> 0) & 0xff;
var b2 = (value >> 8) & 0xff;
var b3 = (value >> 16) & 0xff;
var b4 = (value >> 24) & 0xff;
return b1 << 24 | b2 << 16 | b3 << 8 | b4 << 0;
}

Declare this class:
using static System.Net.IPAddress;
namespace BigEndianExtension
{
public static class BigEndian
{
public static short ToBigEndian(this short value) => HostToNetworkOrder(value);
public static int ToBigEndian(this int value) => HostToNetworkOrder(value);
public static long ToBigEndian(this long value) => HostToNetworkOrder(value);
public static short FromBigEndian(this short value) => NetworkToHostOrder(value);
public static int FromBigEndian(this int value) => NetworkToHostOrder(value);
public static long FromBigEndian(this long value) => NetworkToHostOrder(value);
}
}
Example, create a form with a button and a multiline textbox:
using BigEndianExtension;
private void button1_Click(object sender, EventArgs e)
{
short int16 = 0x1234;
int int32 = 0x12345678;
long int64 = 0x123456789abcdef0;
string text = string.Format("LE:{0:X4}\r\nBE:{1:X4}\r\n", int16, int16.ToBigEndian());
text += string.Format("LE:{0:X8}\r\nBE:{1:X8}\r\n", int32, int32.ToBigEndian());
text += string.Format("LE:{0:X16}\r\nBE:{1:X16}\r\n", int64, int64.ToBigEndian());
textBox1.Text = text;
}
//Some code...

The most straightforward way is to use the BinaryPrimitives.ReadInt32BigEndian(ReadOnlySpan) Method introduced in .NET Standard 2.1
var number = BinaryPrimitives.ReadInt32BigEndian(waveData[297..291]);

If you won't ever again need that reversed, temporary array, you could just create it as you pass the parameter, instead of making four assignments. For example:
int i = 287;
int value = BitConverter.ToInt32({
waveData(i + 3),
waveData(i + 2),
waveData(i + 1),
waveData(i)
}, 0);

I use the following helper functions
public static Int16 ToInt16(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt16(BitConverter.IsLittleEndian ? data.Skip(offset).Take(2).Reverse().ToArray() : data, 0);
return BitConverter.ToInt16(data, offset);
}
public static Int32 ToInt32(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt32(BitConverter.IsLittleEndian ? data.Skip(offset).Take(4).Reverse().ToArray() : data, 0);
return BitConverter.ToInt32(data, offset);
}
public static Int64 ToInt64(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt64(BitConverter.IsLittleEndian ? data.Skip(offset).Take(8).Reverse().ToArray() : data, 0);
return BitConverter.ToInt64(data, offset);
}

You can also use Jon Skeet "Misc Utils" library, available at https://jonskeet.uk/csharp/miscutil/
His library has many utility functions. For Big/Little endian conversions you can check the MiscUtil/Conversion/EndianBitConverter.cs file.
var littleEndianBitConverter = new MiscUtil.Conversion.LittleEndianBitConverter();
littleEndianBitConverter.ToInt64(bytes, offset);
var bigEndianBitConverter = new MiscUtil.Conversion.BigEndianBitConverter();
bigEndianBitConverter.ToInt64(bytes, offset);
His software is from 2009 but I guess it's still relevant.

I dislike BitConverter, because (as Marc Gravell answered) it is specced to rely on system endianness, meaning you technically have to do a system endianness check every time you use BitConverter to ensure you don't have to reverse the array. And usually, with saved files, you generally know the endianness you're trying to read, and that might not be the same. You might just be handling file formats with big-endian values, too, like, for instance, PNG chunks.
Because of that, I just wrote my own methods for this, which take a byte array, the read offset and read length as arguments, as well as a boolean to specify the endianness handling, and which uses bit shifting for efficiency:
public static UInt64 ReadIntFromByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to read a " + bytes + "-byte value at offset " + startIndex + ".");
UInt64 value = 0;
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
value |= (((UInt64)data[offs]) << (8 * index));
}
return value;
}
This code can handle any value between 1 and 8 bytes, both little-endian and big-endian. The only small usage peculiarity is that you need to both give the amount of bytes to read, and need to specifically cast the result to the type you want.
Example from some code where I used it to read the header of some proprietary image type:
Int16 imageWidth = (Int16) ReadIntFromByteArray(fileData, hdrOffset, 2, true);
Int16 imageHeight = (Int16) ReadIntFromByteArray(fileData, hdrOffset + 2, 2, true);
This will read two consecutive 16-bit integers off an array, as signed little-endian values. You can of course just make a bunch of overload functions for all possibilities, like this:
public Int16 ReadInt16FromByteArrayLe(Byte[] data, Int32 startIndex)
{
return (Int16) ReadIntFromByteArray(data, startIndex, 2, true);
}
But personally I didn't bother with that.
And, here's the same for writing bytes:
public static void WriteIntToByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian, UInt64 value)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to write a " + bytes + "-byte value at offset " + startIndex + ".");
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
data[offs] = (Byte) (value >> (8*index) & 0xFF);
}
}
The only requirement here is that you have to cast the input arg to 64-bit unsigned integer when passing it to the function.

public static unsafe int Reverse(int value)
{
byte* p = (byte*)&value;
return (*p << 24) | (p[1] << 16) | (p[2] << 8) | p[3];
}
If unsafe is allowed... Based on Marc Gravell's post

This will reverse the data inline if unsafe code is allowed...
fixed (byte* wavepointer = waveData)
new Span<byte>(wavepointer + offset, 4).Reverse();

How can I improve performance for converting byte[] to int?

I'm writing a Binary file converter in which I need to convert 1-6 byte arrays into int (short-long) values. At the moment I'm using following three functions, I want to know is there anyway to improve the performance?
private string byteToShortParse(byte[] recordData, int offset, int length)
{
byte[] workingSet = new byte[2];
Buffer.BlockCopy(recordData, offset, workingSet, 0, length);
return (BitConverter.ToInt16(workingSet, 0).ToString());
}
private string byteToIntParse(byte[] recordData, int offset, int length)
{
byte[] workingSet = new byte[4];
Buffer.BlockCopy(recordData, offset, workingSet, 0, length);
return (BitConverter.ToInt32(workingSet, 0).ToString());
}
private string byteToLongParse(byte[] recordData, int offset, int length)
{
byte[] workingSet = new byte[8];
Buffer.BlockCopy(recordData, offset, workingSet, 0, length);
return (BitConverter.ToInt32(workingSet, 0).ToString());
}

Edit2:
I suppose if the number of bytes you need to convert to int is variable length (which does seem strange), I suggest doing it this way:
private string bytesToIntParse(byte[] recordData, int offset, int length)
{
long result = 0;
for (int i = 0; i < length; ++i)
{
result |= ((long)recordData[i + offset]) << (i * 8);
}
return result.ToString();
}
Now you have one function, no Buffer.BlockCopy and it supports any length.
Edit1:
You could use unsafe code such as:
// I don't think you need to specify a length parameter, since int32 is always 4 bytes
private string byteToIntParse(byte[] recordData, int offset, int length)
{
unsafe
{
fixed (byte* p = &recordData[offset])
{
// This result will differ on little and big endian architectures.
return (*(int*)p).ToString();
}
}
}
But this is what BitConverter does internally, so I don't think you will gain any performance
Why are you copying the bytes into workingSet? You could just:
return BitConverter.ToInt32(recordData, offset).ToString()
I guess that yields a performance boost since you don't have to call Buffer.BlockCopy every time :P

Yes, optimal variant would be
private string byteToShortParse(byte[] recordData, int offset, int length)
{
if (length == 2)
{
short i = (recordData[offset + 1] << 8) | recordData[offset];
return i.ToString;
} else return "";
}
The same applies to 4-byte and 8-byte values (just more shifts are needed).

how to convert a value type to byte[] in C#?

I want to do the equivalent of this:
byte[] byteArray;
enum commands : byte {one, two};
commands content = one;
byteArray = (byte*)&content;
yes, it's a byte now, but consider I want to change it in the future? how do I make byteArray contain content? (I don't care to copy it).

To convert any value types (not just primitive types) to byte arrays and vice versa:
public T FromByteArray<T>(byte[] rawValue)
{
GCHandle handle = GCHandle.Alloc(rawValue, GCHandleType.Pinned);
T structure = (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
handle.Free();
return structure;
}
public byte[] ToByteArray(object value, int maxLength)
{
int rawsize = Marshal.SizeOf(value);
byte[] rawdata = new byte[rawsize];
GCHandle handle =
GCHandle.Alloc(rawdata,
GCHandleType.Pinned);
Marshal.StructureToPtr(value,
handle.AddrOfPinnedObject(),
false);
handle.Free();
if (maxLength < rawdata.Length) {
byte[] temp = new byte[maxLength];
Array.Copy(rawdata, temp, maxLength);
return temp;
} else {
return rawdata;
}
}

The BitConverter class might be what you are looking for. Example:
int input = 123;
byte[] output = BitConverter.GetBytes(input);
If your enum was known to be an Int32 derived type, you could simply cast its values first:
BitConverter.GetBytes((int)commands.one);

You can use the BitConverter.GetBytes method to do this.

For anyone who's interested in how it works without using BitConverter you can do it like this:
// Convert double to byte[]
public unsafe byte[] pack(double d) {
byte[] packed = new byte[8]; // There are 8 bytes in a double
void* ptr = &d; // Get a reference to the memory containing the double
for (int i = 0; i < 8; i++) { // Each one of the 8 bytes needs to be added to the byte array
packed[i] = (byte)(*(UInt64 *)ptr >> (8 * i)); // Bit shift so that each chunk of 8 bits (1 byte) is cast as a byte and added to array
}
return packed;
}
// Convert byte[] to double
public unsafe double unpackDouble(byte[] data) {
double unpacked = 0.0; // Prepare a chunk of memory ready for the double
void* ptr = &unpacked; // Reference the double memory
for (int i = 0; i < data.Length; i++) {
*(UInt64 *)ptr |= ((UInt64)data[i] << (8 * i)); // Get the bits into the right place and OR into the double
}
return unpacked;
}
In reality it's much easier and safer to use BitConverter but it's fun to know!

C# unsafe value type array to byte array conversions

I use an extension method to convert float arrays into byte arrays:
public static unsafe byte[] ToByteArray(this float[] floatArray, int count)
{
int arrayLength = floatArray.Length > count ? count : floatArray.Length;
byte[] byteArray = new byte[4 * arrayLength];
fixed (float* floatPointer = floatArray)
{
fixed (byte* bytePointer = byteArray)
{
float* read = floatPointer;
float* write = (float*)bytePointer;
for (int i = 0; i < arrayLength; i++)
{
*write++ = *read++;
}
}
}
return byteArray;
}
I understand that an array is a pointer to memory associated with information on the type and number of elements. Also, it seems to me that there is no way of doing a conversion from and to a byte array without copying the data as above.
Have I understood this? Would it even be impossible to write IL to create an array from a pointer, type and length without copying data?
EDIT: Thanks for the answers, I learned some fundamentals and got to try out new tricks!
After initially accepting Davy Landman's answer I found out that while his brilliant StructLayout hack does convert byte arrays into float arrays, it does not work the other way around. To demonstrate:
[StructLayout(LayoutKind.Explicit)]
struct UnionArray
{
[FieldOffset(0)]
public Byte[] Bytes;
[FieldOffset(0)]
public float[] Floats;
}
static void Main(string[] args)
{
// From bytes to floats - works
byte[] bytes = { 0, 1, 2, 4, 8, 16, 32, 64 };
UnionArray arry = new UnionArray { Bytes = bytes };
for (int i = 0; i < arry.Bytes.Length / 4; i++)
Console.WriteLine(arry.Floats[i]);
// From floats to bytes - index out of range
float[] floats = { 0.1f, 0.2f, 0.3f };
arry = new UnionArray { Floats = floats };
for (int i = 0; i < arry.Floats.Length * 4; i++)
Console.WriteLine(arry.Bytes[i]);
}
It seems that the CLR sees both arrays as having the same length. If the struct is created from float data, the byte array's length is just too short.

You can use a really ugly hack to temporary change your array to byte[] using memory manipulation.
This is really fast and efficient as it doesn't require cloning the data and iterating on it.
I tested this hack in both 32 & 64 bit OS, so it should be portable.
The source + sample usage is maintained at https://gist.github.com/1050703 , but for your convenience I'll paste it here as well:
public static unsafe class FastArraySerializer
{
[StructLayout(LayoutKind.Explicit)]
private struct Union
{
[FieldOffset(0)] public byte[] bytes;
[FieldOffset(0)] public float[] floats;
}
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private struct ArrayHeader
{
public UIntPtr type;
public UIntPtr length;
}
private static readonly UIntPtr BYTE_ARRAY_TYPE;
private static readonly UIntPtr FLOAT_ARRAY_TYPE;
static FastArraySerializer()
{
fixed (void* pBytes = new byte[1])
fixed (void* pFloats = new float[1])
{
BYTE_ARRAY_TYPE = getHeader(pBytes)->type;
FLOAT_ARRAY_TYPE = getHeader(pFloats)->type;
}
}
public static void AsByteArray(this float[] floats, Action<byte[]> action)
{
if (floats.handleNullOrEmptyArray(action))
return;
var union = new Union {floats = floats};
union.floats.toByteArray();
try
{
action(union.bytes);
}
finally
{
union.bytes.toFloatArray();
}
}
public static void AsFloatArray(this byte[] bytes, Action<float[]> action)
{
if (bytes.handleNullOrEmptyArray(action))
return;
var union = new Union {bytes = bytes};
union.bytes.toFloatArray();
try
{
action(union.floats);
}
finally
{
union.floats.toByteArray();
}
}
public static bool handleNullOrEmptyArray<TSrc,TDst>(this TSrc[] array, Action<TDst[]> action)
{
if (array == null)
{
action(null);
return true;
}
if (array.Length == 0)
{
action(new TDst[0]);
return true;
}
return false;
}
private static ArrayHeader* getHeader(void* pBytes)
{
return (ArrayHeader*)pBytes - 1;
}
private static void toFloatArray(this byte[] bytes)
{
fixed (void* pArray = bytes)
{
var pHeader = getHeader(pArray);
pHeader->type = FLOAT_ARRAY_TYPE;
pHeader->length = (UIntPtr)(bytes.Length / sizeof(float));
}
}
private static void toByteArray(this float[] floats)
{
fixed(void* pArray = floats)
{
var pHeader = getHeader(pArray);
pHeader->type = BYTE_ARRAY_TYPE;
pHeader->length = (UIntPtr)(floats.Length * sizeof(float));
}
}
}
And the usage is:
var floats = new float[] {0, 1, 0, 1};
floats.AsByteArray(bytes =>
{
foreach (var b in bytes)
{
Console.WriteLine(b);
}
});

Yes, the type information and data is in the same memory block, so that is impossible unless you overwrite the type information in a float array to fool the system that it's byte array. That would be a really ugly hack, and could easily blow up...
Here's how you can convert the floats without unsafe code if you like:
public static byte[] ToByteArray(this float[] floatArray) {
int len = floatArray.Length * 4;
byte[] byteArray = new byte[len];
int pos = 0;
foreach (float f in floatArray) {
byte[] data = BitConverter.GetBytes(f);
Array.Copy(data, 0, byteArray, pos, 4);
pos += 4;
}
return byteArray;
}

This question is the reverse of What is the fastest way to convert a float[] to a byte[]?.
I've answered with a union kind of hack to skip the whole copying of the data. You could easily reverse this (length = length *sizeof(Double).

I've written something similar for quick conversion between arrays. It's basically an ugly proof-of-concept more than a handsome solution. ;)
public static TDest[] ConvertArray<TSource, TDest>(TSource[] source)
where TSource : struct
where TDest : struct {
if (source == null)
throw new ArgumentNullException("source");
var sourceType = typeof(TSource);
var destType = typeof(TDest);
if (sourceType == typeof(char) || destType == typeof(char))
throw new NotSupportedException(
"Can not convert from/to a char array. Char is special " +
"in a somewhat unknown way (like enums can't be based on " +
"char either), and Marshal.SizeOf returns 1 even when the " +
"values held by a char can be above 255."
);
var sourceByteSize = Buffer.ByteLength(source);
var destTypeSize = Marshal.SizeOf(destType);
if (sourceByteSize % destTypeSize != 0)
throw new Exception(
"The source array is " + sourceByteSize + " bytes, which can " +
"not be transfered to chunks of " + destTypeSize + ", the size " +
"of type " + typeof(TDest).Name + ". Change destination type or " +
"pad the source array with additional values."
);
var destCount = sourceByteSize / destTypeSize;
var destArray = new TDest[destCount];
Buffer.BlockCopy(source, 0, destArray, 0, sourceByteSize);
return destArray;
}
}

public byte[] ToByteArray(object o)
{
int size = Marshal.SizeOf(o);
byte[] buffer = new byte[size];
IntPtr p = Marshal.AllocHGlobal(size);
try
{
Marshal.StructureToPtr(o, p, false);
Marshal.Copy(p, buffer, 0, size);
}
finally
{
Marshal.FreeHGlobal(p);
}
return buffer;
}
this may help you to convert an object to a byte array.

You should check my answer to a similar question: What is the fastest way to convert a float[] to a byte[]?.
In it you'll find portable code (32/64 bit compatible) to let you view a float array as a byte array or vice-versa, without copying the data. It's the fastest way that I know of to do such thing.
If you're just interested in the code, it's maintained at https://gist.github.com/1050703 .

Well - if you still interested in that hack - check out this modified code - it works like a charm and costs ~0 time, but it may not work in future since it's a hack allowing to gain full access to the whole process address space without trust requirements and unsafe marks.
[StructLayout(LayoutKind.Explicit)]
struct ArrayConvert
{
public static byte[] GetBytes(float[] floats)
{
ArrayConvert ar = new ArrayConvert();
ar.floats = floats;
ar.length.val = floats.Length * 4;
return ar.bytes;
}
public static float[] GetFloats(byte[] bytes)
{
ArrayConvert ar = new ArrayConvert();
ar.bytes = bytes;
ar.length.val = bytes.Length / 4;
return ar.floats;
}
public static byte[] GetTop4BytesFrom(object obj)
{
ArrayConvert ar = new ArrayConvert();
ar.obj = obj;
return new byte[]
{
ar.top4bytes.b0,
ar.top4bytes.b1,
ar.top4bytes.b2,
ar.top4bytes.b3
};
}
public static byte[] GetBytesFrom(object obj, int size)
{
ArrayConvert ar = new ArrayConvert();
ar.obj = obj;
ar.length.val = size;
return ar.bytes;
}
class ArrayLength
{
public int val;
}
class Top4Bytes
{
public byte b0;
public byte b1;
public byte b2;
public byte b3;
}
[FieldOffset(0)]
private Byte[] bytes;
[FieldOffset(0)]
private object obj;
[FieldOffset(0)]
private float[] floats;
[FieldOffset(0)]
private ArrayLength length;
[FieldOffset(0)]
private Top4Bytes top4bytes;
}