Related
I've got a super simple program. My intention is to copy standard input to standard output. Here's the source code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace LALA
{
class LALA
{
static void Main()
{
int bufferSize = 40;
Console.OpenStandardInput().CopyTo(Console.OpenStandardOutput(), bufferSize);
}
}
}
If I set bufferSize to 40, then for any input I just see 'C o n', 'C o n s', or 'C o n s o l e' etc.
If I set bufferSize to 41, then everything is fine.
My question is am I doing something wrong, and by accident values above 41 work?
To clarify, this is the output when I type asd:
c:\some_path>ConsoleApplication2.exe
asd
C o n
Surprise! This appears to be internal buffer overflow and I'll share what I found, though it's not yet a real proof it makes sense to me.
The code for CopyTo method of the Stream class is: (taken with reflector)
public void CopyTo(Stream destination, int bufferSize)
{
//bunch of non relevant validations...
this.InternalCopyTo(destination, bufferSize);
}
Now the code for InternalCopyTo is:
private void InternalCopyTo(Stream destination, int bufferSize)
{
byte[] array = new byte[bufferSize];
int count;
while ((count = this.Read(array, 0, array.Length)) != 0)
{
destination.Write(array, 0, count);
}
}
The console stream instance is of type __ConsoleStream (sealed internal class in System.IO) and its Read method code:
public override int Read([In] [Out] byte[] buffer, int offset, int count)
{
//bunch of non relevant validations...
int errorCode = 0;
int num = __ConsoleStream.ReadFileNative(this._handle, buffer, offset, count, 0, out errorCode);
if (num == -1)
{
__Error.WinIOError(errorCode, string.Empty);
}
return num;
}
And finally ReadFileNative code of __ConsoleStream:
private unsafe static int ReadFileNative(SafeFileHandle hFile, byte[] bytes, int offset, int count, int mustBeZero, out int errorCode)
{
if (bytes.Length - offset < count)
{
throw new IndexOutOfRangeException(Environment.GetResourceString("IndexOutOfRange_IORaceCondition"));
}
if (bytes.Length == 0)
{
errorCode = 0;
return 0;
}
__ConsoleStream.WaitForAvailableConsoleInput(hFile);
int result;
int num;
fixed (byte* ptr = bytes)
{
num = __ConsoleStream.ReadFile(hFile, ptr + (IntPtr)offset / 1, count, out result, Win32Native.NULL);
}
if (num != 0)
{
errorCode = 0;
return result;
}
errorCode = Marshal.GetLastWin32Error();
if (errorCode == 109)
{
return 0;
}
return -1;
}
The ReadFile method is low level call:
[DllImport("kernel32.dll", SetLastError = true)]
private unsafe static extern int ReadFile(SafeFileHandle handle, byte* bytes, int numBytesToRead, out int numBytesRead, IntPtr mustBeZero);
My assumption at this point is that behind the scenes, 40 bytes are "reserved" somewhere to internal data so if the buffer is not more than that, you will see that reserved data which in this case of console application is the process name.
I will keep investigating this when I'll have more time and try to reproduce, this case is quite special since both streams point to the same "file" so you can write to it while reading it.
I am making application in C# which has a byte array containing hex values.
I am getting data as a big-endian but I want it as a little-endian and I am using Bitconverter.toInt32 method for converting that value to integer.
My problem is that before converting the value, I have to copy that 4 byte data into temporary array from source byte array and then reverse that temporary byte array.
I can't reverse source array because it also contains other data.
Because of that my application becomes slow.
In the code I have one source array of byte as waveData[] which contains a lot of data.
byte[] tempForTimestamp=new byte[4];
tempForTimestamp[0] = waveData[290];
tempForTimestamp[1] = waveData[289];
tempForTimestamp[2] = waveData[288];
tempForTimestamp[3] = waveData[287];
int number = BitConverter.ToInt32(tempForTimestamp, 0);
Is there any other method for that conversion?
Add a reference to System.Memory nuget and use BinaryPrimitives.ReverseEndianness().
using System.Buffers.Binary;
number = BinaryPrimitives.ReverseEndianness(number);
It supports both signed and unsigned integers (byte/short/int/long).
In modern-day Linq the one-liner and easiest to understand version would be:
int number = BitConverter.ToInt32(waveData.Skip(286).Take(4).Reverse().ToArray(), 0);
You could also...
byte[] tempForTimestamp = new byte[4];
Array.Copy(waveData, 287, tempForTimestamp, 0, 4);
Array.Reverse(tempForTimestamp);
int number = BitConverter.ToInt32(tempForTimestamp);
:)
If you know the data is big-endian, perhaps just do it manually:
int value = (buffer[i++] << 24) | (buffer[i++] << 16)
| (buffer[i++] << 8) | buffer[i++];
this will work reliably on any CPU, too. Note i is your current offset into the buffer.
Another approach would be to shuffle the array:
byte tmp = buffer[i+3];
buffer[i+3] = buffer[i];
buffer[i] = tmp;
tmp = buffer[i+2];
buffer[i+2] = buffer[i+1];
buffer[i+1] = tmp;
int value = BitConverter.ToInt32(buffer, i);
i += 4;
I find the first immensely more readable, and there are no branches / complex code, so it should work pretty fast too. The second could also run into problems on some platforms (where the CPU is already running big-endian).
Here you go
public static int SwapEndianness(int value)
{
var b1 = (value >> 0) & 0xff;
var b2 = (value >> 8) & 0xff;
var b3 = (value >> 16) & 0xff;
var b4 = (value >> 24) & 0xff;
return b1 << 24 | b2 << 16 | b3 << 8 | b4 << 0;
}
Declare this class:
using static System.Net.IPAddress;
namespace BigEndianExtension
{
public static class BigEndian
{
public static short ToBigEndian(this short value) => HostToNetworkOrder(value);
public static int ToBigEndian(this int value) => HostToNetworkOrder(value);
public static long ToBigEndian(this long value) => HostToNetworkOrder(value);
public static short FromBigEndian(this short value) => NetworkToHostOrder(value);
public static int FromBigEndian(this int value) => NetworkToHostOrder(value);
public static long FromBigEndian(this long value) => NetworkToHostOrder(value);
}
}
Example, create a form with a button and a multiline textbox:
using BigEndianExtension;
private void button1_Click(object sender, EventArgs e)
{
short int16 = 0x1234;
int int32 = 0x12345678;
long int64 = 0x123456789abcdef0;
string text = string.Format("LE:{0:X4}\r\nBE:{1:X4}\r\n", int16, int16.ToBigEndian());
text += string.Format("LE:{0:X8}\r\nBE:{1:X8}\r\n", int32, int32.ToBigEndian());
text += string.Format("LE:{0:X16}\r\nBE:{1:X16}\r\n", int64, int64.ToBigEndian());
textBox1.Text = text;
}
//Some code...
The most straightforward way is to use the BinaryPrimitives.ReadInt32BigEndian(ReadOnlySpan) Method introduced in .NET Standard 2.1
var number = BinaryPrimitives.ReadInt32BigEndian(waveData[297..291]);
If you won't ever again need that reversed, temporary array, you could just create it as you pass the parameter, instead of making four assignments. For example:
int i = 287;
int value = BitConverter.ToInt32({
waveData(i + 3),
waveData(i + 2),
waveData(i + 1),
waveData(i)
}, 0);
I use the following helper functions
public static Int16 ToInt16(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt16(BitConverter.IsLittleEndian ? data.Skip(offset).Take(2).Reverse().ToArray() : data, 0);
return BitConverter.ToInt16(data, offset);
}
public static Int32 ToInt32(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt32(BitConverter.IsLittleEndian ? data.Skip(offset).Take(4).Reverse().ToArray() : data, 0);
return BitConverter.ToInt32(data, offset);
}
public static Int64 ToInt64(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
return BitConverter.ToInt64(BitConverter.IsLittleEndian ? data.Skip(offset).Take(8).Reverse().ToArray() : data, 0);
return BitConverter.ToInt64(data, offset);
}
You can also use Jon Skeet "Misc Utils" library, available at https://jonskeet.uk/csharp/miscutil/
His library has many utility functions. For Big/Little endian conversions you can check the MiscUtil/Conversion/EndianBitConverter.cs file.
var littleEndianBitConverter = new MiscUtil.Conversion.LittleEndianBitConverter();
littleEndianBitConverter.ToInt64(bytes, offset);
var bigEndianBitConverter = new MiscUtil.Conversion.BigEndianBitConverter();
bigEndianBitConverter.ToInt64(bytes, offset);
His software is from 2009 but I guess it's still relevant.
I dislike BitConverter, because (as Marc Gravell answered) it is specced to rely on system endianness, meaning you technically have to do a system endianness check every time you use BitConverter to ensure you don't have to reverse the array. And usually, with saved files, you generally know the endianness you're trying to read, and that might not be the same. You might just be handling file formats with big-endian values, too, like, for instance, PNG chunks.
Because of that, I just wrote my own methods for this, which take a byte array, the read offset and read length as arguments, as well as a boolean to specify the endianness handling, and which uses bit shifting for efficiency:
public static UInt64 ReadIntFromByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to read a " + bytes + "-byte value at offset " + startIndex + ".");
UInt64 value = 0;
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
value |= (((UInt64)data[offs]) << (8 * index));
}
return value;
}
This code can handle any value between 1 and 8 bytes, both little-endian and big-endian. The only small usage peculiarity is that you need to both give the amount of bytes to read, and need to specifically cast the result to the type you want.
Example from some code where I used it to read the header of some proprietary image type:
Int16 imageWidth = (Int16) ReadIntFromByteArray(fileData, hdrOffset, 2, true);
Int16 imageHeight = (Int16) ReadIntFromByteArray(fileData, hdrOffset + 2, 2, true);
This will read two consecutive 16-bit integers off an array, as signed little-endian values. You can of course just make a bunch of overload functions for all possibilities, like this:
public Int16 ReadInt16FromByteArrayLe(Byte[] data, Int32 startIndex)
{
return (Int16) ReadIntFromByteArray(data, startIndex, 2, true);
}
But personally I didn't bother with that.
And, here's the same for writing bytes:
public static void WriteIntToByteArray(Byte[] data, Int32 startIndex, Int32 bytes, Boolean littleEndian, UInt64 value)
{
Int32 lastByte = bytes - 1;
if (data.Length < startIndex + bytes)
throw new ArgumentOutOfRangeException("startIndex", "Data array is too small to write a " + bytes + "-byte value at offset " + startIndex + ".");
for (Int32 index = 0; index < bytes; index++)
{
Int32 offs = startIndex + (littleEndian ? index : lastByte - index);
data[offs] = (Byte) (value >> (8*index) & 0xFF);
}
}
The only requirement here is that you have to cast the input arg to 64-bit unsigned integer when passing it to the function.
public static unsafe int Reverse(int value)
{
byte* p = (byte*)&value;
return (*p << 24) | (p[1] << 16) | (p[2] << 8) | p[3];
}
If unsafe is allowed... Based on Marc Gravell's post
This will reverse the data inline if unsafe code is allowed...
fixed (byte* wavepointer = waveData)
new Span<byte>(wavepointer + offset, 4).Reverse();
I am currently working on a network tool that needs to decode/encode a particular protocol that packs fields into dense bit arrays at arbitrary positions. For example, one part of the protocol uses 3 bytes to represent a number of different fields:
Bit Position(s) Length (In Bits) Type
0 1 bool
1-5 5 int
6-13 8 int
14-22 9 uint
23 1 bool
As you can see, several of the fields span multiple bytes. Many (most) are also shorter than the built-in type that might be used to represent them, such as the first int field which is only 5 bits long. In these cases, the most significant bits of the target type (such as an Int32 or Int16) should be padded with 0 to make up the difference.
My problem is that I am having a difficult time processing this kind of data. Specifically, I am having a hard time figuring out how to efficiently get arbitrary length bit arrays, populate them with the appropriate bits from the source buffer, pad them to match the target type, and convert the padded bit arrays to the target type. In an ideal world, I would be able to take the byte[3] in the example above and call a method like GetInt32(byte[] bytes, int startBit, int length).
The closest thing in the wild that I've found is a BitStream class, but it appears to want individual values to line up on byte/word boundaries (and the half-streaming/half-indexed access convention of the class makes it a little confusing).
My own first attempt was to use the BitArray class, but that proved somewhat unwieldy. It's easy enough to stuff all the bits from the buffer into a large BitArray, transfer only the ones you want from the source BitArray to a new temporary BitArray, and then convert that into the target value...but it seems wrong, and very time consuming.
I am now considering a class like the following that references (or creates) a source/target byte[] buffer along with an offset and provides get and set methods for certain target types. The tricky part is that getting/setting values may span multiple bytes.
class BitField
{
private readonly byte[] _bytes;
private readonly int _offset;
public BitField(byte[] bytes)
: this(bytes, 0)
{
}
public BitField(byte[] bytes, int offset)
{
_bytes = bytes;
_offset = offset;
}
public BitField(int size)
: this(new byte[size], 0)
{
}
public bool this[int bit]
{
get { return IsSet(bit); }
set { if (value) Set(bit); else Clear(bit); }
}
public bool IsSet(int bit)
{
return (_bytes[_offset + (bit / 8)] & (1 << (bit % 8))) != 0;
}
public void Set(int bit)
{
_bytes[_offset + (bit / 8)] |= unchecked((byte)(1 << (bit % 8)));
}
public void Clear(int bit)
{
_bytes[_offset + (bit / 8)] &= unchecked((byte)~(1 << (bit % 8)));
}
//startIndex = the index of the bit at which to start fetching the value
//length = the number of bits to include - may be less than 32 in which case
//the most significant bits of the target type should be padded with 0
public int GetInt32(int startIndex, int length)
{
//NEED CODE HERE
}
//startIndex = the index of the bit at which to start storing the value
//length = the number of bits to use, if less than the number of bits required
//for the source type, precision may be lost
//value = the value to store
public void SetValue(int startIndex, int length, int value)
{
//NEED CODE HERE
}
//Other Get.../Set... methods go here
}
I am looking for any guidance in this area such as third-party libraries, algorithms for getting/setting values at arbitrary bit positions that span multiple bytes, feedback on my approach, etc. I included the class above for clarification and am not necessarily looking for code to fill it in (though I won't argue if someone wants to work it out!).
As promised, here is the class I ended up creating for this purpose. It will wrap an arbitrary byte array at an optionally specified index and allowing reading/writing at the bit level. It provides methods for reading/writing arbitrary blocks of bits from other byte arrays or for reading/writing primitive values with user-defined offsets and lengths. It works very well for my situation and solves the exact question I asked above. However, it does have a couple shortcomings. The first is that it is obviously not greatly documented - I just haven't had the time. The second is that there are no bounds or other checks. It also currently requires the MiscUtil library to provide endian conversion. All that said, hopefully this can help solve or serve as a starting point for someone else with a similar use case.
internal class BitField
{
private readonly byte[] _bytes;
private readonly int _offset;
private EndianBitConverter _bitConverter = EndianBitConverter.Big;
public BitField(byte[] bytes)
: this(bytes, 0)
{
}
//offset = the offset (in bytes) into the wrapped byte array
public BitField(byte[] bytes, int offset)
{
_bytes = bytes;
_offset = offset;
}
public BitField(int size)
: this(new byte[size], 0)
{
}
//fill == true = initially set all bits to 1
public BitField(int size, bool fill)
: this(new byte[size], 0)
{
if (!fill) return;
for(int i = 0 ; i < size ; i++)
{
_bytes[i] = 0xff;
}
}
public byte[] Bytes
{
get { return _bytes; }
}
public int Offset
{
get { return _offset; }
}
public EndianBitConverter BitConverter
{
get { return _bitConverter; }
set { _bitConverter = value; }
}
public bool this[int bit]
{
get { return IsBitSet(bit); }
set { if (value) SetBit(bit); else ClearBit(bit); }
}
public bool IsBitSet(int bit)
{
return (_bytes[_offset + (bit / 8)] & (1 << (7 - (bit % 8)))) != 0;
}
public void SetBit(int bit)
{
_bytes[_offset + (bit / 8)] |= unchecked((byte)(1 << (7 - (bit % 8))));
}
public void ClearBit(int bit)
{
_bytes[_offset + (bit / 8)] &= unchecked((byte)~(1 << (7 - (bit % 8))));
}
//index = the index of the source BitField at which to start getting bits
//length = the number of bits to get
//size = the total number of bytes required (0 for arbitrary length return array)
//fill == true = set all padding bits to 1
public byte[] GetBytes(int index, int length, int size, bool fill)
{
if(size == 0) size = (length + 7) / 8;
BitField bitField = new BitField(size, fill);
for(int s = index, d = (size * 8) - length ; s < index + length && d < (size * 8) ; s++, d++)
{
bitField[d] = IsBitSet(s);
}
return bitField._bytes;
}
public byte[] GetBytes(int index, int length, int size)
{
return GetBytes(index, length, size, false);
}
public byte[] GetBytes(int index, int length)
{
return GetBytes(index, length, 0, false);
}
//bytesIndex = the index (in bits) into the bytes array at which to start copying
//index = the index (in bits) in this BitField at which to put the value
//length = the number of bits to copy from the bytes array
public void SetBytes(byte[] bytes, int bytesIndex, int index, int length)
{
BitField bitField = new BitField(bytes);
for (int i = 0; i < length; i++)
{
this[index + i] = bitField[bytesIndex + i];
}
}
public void SetBytes(byte[] bytes, int index, int length)
{
SetBytes(bytes, 0, index, length);
}
public void SetBytes(byte[] bytes, int index)
{
SetBytes(bytes, 0, index, bytes.Length * 8);
}
//UInt16
//index = the index (in bits) at which to start getting the value
//length = the number of bits to use for the value, if less than required the value is padded with 0
public ushort GetUInt16(int index, int length)
{
return _bitConverter.ToUInt16(GetBytes(index, length, 2), 0);
}
public ushort GetUInt16(int index)
{
return GetUInt16(index, 16);
}
//valueIndex = the index (in bits) of the value at which to start copying
//index = the index (in bits) in this BitField at which to put the value
//length = the number of bits to copy from the value
public void Set(ushort value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(ushort value, int index)
{
Set(value, 0, index, 16);
}
//UInt32
public uint GetUInt32(int index, int length)
{
return _bitConverter.ToUInt32(GetBytes(index, length, 4), 0);
}
public uint GetUInt32(int index)
{
return GetUInt32(index, 32);
}
public void Set(uint value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(uint value, int index)
{
Set(value, 0, index, 32);
}
//UInt64
public ulong GetUInt64(int index, int length)
{
return _bitConverter.ToUInt64(GetBytes(index, length, 8), 0);
}
public ulong GetUInt64(int index)
{
return GetUInt64(index, 64);
}
public void Set(ulong value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(ulong value, int index)
{
Set(value, 0, index, 64);
}
//Int16
public short GetInt16(int index, int length)
{
return _bitConverter.ToInt16(GetBytes(index, length, 2, IsBitSet(index)), 0);
}
public short GetInt16(int index)
{
return GetInt16(index, 16);
}
public void Set(short value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(short value, int index)
{
Set(value, 0, index, 16);
}
//Int32
public int GetInt32(int index, int length)
{
return _bitConverter.ToInt32(GetBytes(index, length, 4, IsBitSet(index)), 0);
}
public int GetInt32(int index)
{
return GetInt32(index, 32);
}
public void Set(int value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(int value, int index)
{
Set(value, 0, index, 32);
}
//Int64
public long GetInt64(int index, int length)
{
return _bitConverter.ToInt64(GetBytes(index, length, 8, IsBitSet(index)), 0);
}
public long GetInt64(int index)
{
return GetInt64(index, 64);
}
public void Set(long value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(long value, int index)
{
Set(value, 0, index, 64);
}
//Char
public char GetChar(int index, int length)
{
return _bitConverter.ToChar(GetBytes(index, length, 2), 0);
}
public char GetChar(int index)
{
return GetChar(index, 16);
}
public void Set(char value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(char value, int index)
{
Set(value, 0, index, 16);
}
//Bool
public bool GetBool(int index, int length)
{
return _bitConverter.ToBoolean(GetBytes(index, length, 1), 0);
}
public bool GetBool(int index)
{
return GetBool(index, 8);
}
public void Set(bool value, int valueIndex, int index, int length)
{
SetBytes(_bitConverter.GetBytes(value), valueIndex, index, length);
}
public void Set(bool value, int index)
{
Set(value, 0, index, 8);
}
//Single and double precision floating point values must always use the correct number of bits
public float GetSingle(int index)
{
return _bitConverter.ToSingle(GetBytes(index, 32, 4), 0);
}
public void SetSingle(float value, int index)
{
SetBytes(_bitConverter.GetBytes(value), 0, index, 32);
}
public double GetDouble(int index)
{
return _bitConverter.ToDouble(GetBytes(index, 64, 8), 0);
}
public void SetDouble(double value, int index)
{
SetBytes(_bitConverter.GetBytes(value), 0, index, 64);
}
}
If your packets are always smaller than 8 or 4 bytes it would be easier to store each packet in an Int32 or Int64. The byte array only complicates things. You do have to pay attention to High-Endian vs Low-Endian storage.
And then, for a 3 byte package:
public static void SetValue(Int32 message, int startIndex, int length, int value)
{
// we want lengthx1
int mask = (1 << length) - 1;
value = value & mask; // or check and throw
int offset = 24 - startIndex - length; // 24 = 3 * 8
message = message | (value << offset);
}
First up, it seems you have re invented the wheel with the System.Collections.BitArray class. As for actually finding the value of a specific field of bits, I think that can easily be accomplished with a little math magic of the following pseudocode:
Start at the most distant digit in your selection (startIndex + length).
If it is set, add 2^(distance from digit). In this case, it would be 0 (mostDistance - self = 0). So Add 2^0 (1).
Move one bit to the left.
Repeat for each digit in the length you want.
In that situation, if you had a bit array like so:
10001010
And you want the value of digits 0-3, you would get something like:
[Index 3] [Index 2] [Index 1] [Index 0]
(3 - 3) (3 - 2) (3 - 1) (3 - 0)
=============================================
(0 * 2^0) + (0 * 2^1) + (0 * 2^2) + (1 * 2^3) = 8
Since 1000 (binary) == 8, the math works out.
What's the problem with just using simple bit shifts to get your values out?
int data = Convert.ToInt32( "110000000010000000100001", 2 );
bool v1 = ( data & 1 ) == 1; // True
int v2 = ( data >> 1 ) & 0x1F; // 16
int v3 = ( data >> 6 ) & 0xFF; // 128
uint v4 = (uint )( data >> 14 ) & 0x1FF; // 256
bool v5 = ( data >> 23 ) == 1; // True
This is a pretty good article covering the subject. it's in C, but the same concepts still apply.
I have a byte array, and I would like to read an integer from this array. How can I do it?
Something like this:
int i;
tab = new byte[32];
i = readint(tab,0,3); // i = int from tab[0] to tab[3] (int = 4 bytes?)
i = readint(tab,4,7);
etc...
byte[] bytes = { 0, 0, 0, 25 };
// If the system architecture is little-endian (that is, little end first),
// reverse the byte array.
if (BitConverter.IsLittleEndian)
Array.Reverse(bytes);
int i = BitConverter.ToInt32(bytes, 0);
Console.WriteLine("int: {0}", i);
Ref: How to: Convert a byte Array to an int
Also, there is a class called Endian in Jon Skeet's miscutil library which implements conversion methods between a byte array and various primitive types, taking endianness into account.
For your question, usage would be something like:
// Input data
byte[] tab = new byte[32];
// Pick the appropriate endianness
Endian endian = Endian.Little;
// Use the appropriate endian to convert
int a = endian.ToInt32(tab, 0);
int b = endian.ToInt32(tab, 4);
int c = endian.ToInt32(tab, 8);
int d = endian.ToInt32(tab, 16);
...
A simplified version of the Endian class would be something like:
public abstract class Endian
{
public short ToInt16(byte[] value, int startIndex)
{
return unchecked((short)FromBytes(value, startIndex, 2));
}
public int ToInt32(byte[] value, int startIndex)
{
return unchecked((int)FromBytes(value, startIndex, 4));
}
public long ToInt64(byte[] value, int startIndex)
{
return FromBytes(value, startIndex, 8);
}
// This same method can be used by int16, int32 and int64.
protected virtual long FromBytes(byte[] buffer, int startIndex, int len);
}
And then the FromBytes abstract method is implemented differently for each endian type.
public class BigEndian : Endian
{
protected override long FromBytes(byte[] buffer, int startIndex, int len)
{
long ret = 0;
for (int i=0; i < len; i++)
{
ret = unchecked((ret << 8) | buffer[startIndex+i]);
}
return ret;
}
}
public class LittleEndian : Endian
{
protected override long FromBytes(byte[] buffer, int startIndex, int len)
{
long ret = 0;
for (int i=0; i < len; i++)
{
ret = unchecked((ret << 8) | buffer[startIndex+len-1-i]);
}
return ret;
}
}
You could use BitConverter.ToInt32. Have a look at this.
If you wanted to do it manually, something like that should do the trick!
byte[] data = ...;
int startIndex = 0;
int value = data[startIndex];
for (int i=1;i<4;i++)
{
value <<= 8;
value |= data[i+startIndex];
}
I have the 4 bytes that represent an integer stored in 2 separate byte arrays. I would like to convert these into an Int32 WITHOUT copying to a third byte array and reading that using memorystream.
The reason the data is split across two byte arrays is because this is a simplified example of my issue which involves huge amounts of data that cannot fit into a single bytearray.
Is there any way to achieve this? I do not wish to concatenate the two byte arrays into a thrid because of the performance implications which are critical to me.
Moon
You can use a struct layout like this
[StructLayout(LayoutKind.Explicit, Size=4)]
struct UnionInt32Value
{
[FieldOffset(0)] public byte byte1;
[FieldOffset(1)] public byte byte2;
[FieldOffset(2)] public byte byte3;
[FieldOffset(3)] public byte byte4;
[FieldOffset(0)] public Int32 iVal;
}
Assign your bytes in the correct order then read your Int32 from iVal;
EDIT: Sample code
using System;
using System.Runtime.InteropServices;
namespace Test
{
class Program
{
[StructLayout(LayoutKind.Explicit, Size=4)]
struct UnionInt32Value
{
[FieldOffset(0)] public byte byte1;
[FieldOffset(1)] public byte byte2;
[FieldOffset(2)] public byte byte3;
[FieldOffset(3)] public byte byte4;
[FieldOffset(0)] public Int32 iVal;
}
public static void Main(string[] args)
{
UnionInt32Value v = new UnionInt32Value();
v.byte1=1;
v.byte2=0;
v.byte3=0;
v.byte4=0;
Console.WriteLine("this is one " + v.iVal);
v.byte1=0xff;
v.byte2=0xff;
v.byte3=0xff;
v.byte4=0xff;
Console.WriteLine("this is minus one " + v.iVal);
Console.Write("Press any key to continue . . . ");
Console.ReadKey(true);
}
}
}
Something like this?
int x = (array1[index] << 16) + array2[index];
Of course, you didn't specify a language, but that's the gist of it.
The BitConverter class is intended for this:
byte[] parts = { byte1, byte2, byte3, byte4 };
int value = BitConverter.ToInt32(parts, 0);
You can use BitConverter twice, like:
byte[] bytes0 = new byte[] { 255, 255 };
byte[] bytes1 = new byte[] { 0, 0 };
int res = BitConverter.ToInt16(bytes0, 0) << 16;
res |= BitConverter.ToUInt16(bytes1, 0);
Which yields -65536 (0b11111111 11111111 00000000 00000000)
If your integer parts isn't at position 0 in the array, you just replace the 0 in ToUint16 to change the position.
Little extension method:
public static class BitConverterExt
{
public static int ToInt32(byte[] arr0, int index0, byte[] arr1, int index1)
{
int partRes = BitConverter.ToInt16(arr1, index1) << 16;
return partRes | BitConverter.ToUInt16(arr0, index0);
}
}
Usage:
byte[] bytes0 = new byte[] { 0x0, 0xA };
byte[] bytes1 = new byte[] { 0x64, 0xFF };
int res = BitConverterExt.ToInt32(bytes0, 0, bytes1, 0);
//Res -10221056 (0xFF640A00)
If I understand correctly, you are having a problem whilst reading across the boundary of the two arrays. If that is so, this routine will read an integer anywhere in the two arrays, even if it is across the two of them.
int ReadInteger(byte[] array1, byte[] array2, int offset)
{
if (offset < 0 || (offset + 4) > (array1.Length + array2.Length))
throw new ArgumentOutOfRangeException();
if (offset <= (array1.Length - 4))
return BitConverter.ToInt32(array1, offset);
else if (offset >= array1.Length)
return BitConverter.ToInt32(array2, offset - array1.Length);
else
{
var buffer = new byte[4];
var numFirst = array1.Length - offset;
Array.Copy(array1, offset, buffer, 0, numFirst);
Array.Copy(array2, 0, buffer, numFirst, 4 - numFirst);
return BitConverter.ToInt32(buffer, 0);
}
}
Note: depending on how your integers are stored, you might want to change the order in which bytes are copied.