I am reading a binary log file produced by a piece of equipment.
I have the data in a byte[].
If I need to read two bytes to create a short I can do something like this:
short value = (short)(byte[1] << 8);
value += byte[2];
Now I know the value is the correct for valid data.
How would I know if the file was messed up and lets say the values FF FF were in those two places in the byte array?
When I look at the resultant value of converting FF FF to a short, I get a -1.
Is this a normal value for FF FF?, or did the computer just hit some kind of short bound and roll over with invalid data?
For my purposes all of theses numbers are going to be positive. If FF FF is actually a short -1, then I just need to validate that all my results are postitive.
Thank you,
Keith
BTW, I am also reading other number data types. I'll show them here just because.
The Read function is the basic part of reading from the byte[]. All the other data type reads use the basic Read() function.
public byte Read()
{
//advance position and then return byte at position
byte returnValue;
if (_CurrentPosition < _count - 1)
{
returnValue= _array[_offset + ++_CurrentPosition];
return returnValue;
}
else
throw new System.IO.EndOfStreamException
("Cannot Read Array, at end of stream.");
}
public float ReadFloat()
{
byte[] floatTemp = new byte[4];
for (int i = 3; i >= 0; i--)
{
floatTemp[i] = Read();
}
float returnValue = System.BitConverter.ToSingle
(floatTemp, 0);
if (float.IsNaN(returnValue))
{
throw new Execption("Not a Number");
}
return returnValue;
}
public short ReadInt16()
{
short returnValue = (short)(Read() << 8);
returnValue += Read();
return returnValue;
}
public int ReadInt32()
{
int returnValue = Read() << 24;
returnValue += Read() << 16;
returnValue += Read() << 8;
returnValue += Read();
return returnValue;
}
0xffff (all bits equal to 1) is -1 for signed shorts, yes. Read up on Two's complement to learn more about the details. You can switch to a larger datatype, or (as suggested by Grzenio) just use an unsigned type.
Well, you seemed to have found BitConverter for singles. Now let's see if we can get to to use it for everything else as well...
MemoryStream mem = new MemoryStream(_array);
float ReadFloat(Stream str)
{
byte[] bytes = str.Read(out bytes, 0, 4);
return BitConverter.ToSingle(bytes, 0)
}
public int ReadInt32(Stream str)
{
byte[] bytes = str.Read(out bytes, 0, 4);
return BitConverter.ToInt32(bytes, 0)
}
Did you try to use ushort (unsigned short)?
I think you would be better served using the System.BitConverter class.
Specifically for shorts the ToInt16 method.
You didn't mention it in your question, but you should also make sure that you know what endianess the hardware device is writing its data in. From your examples it looks like the float is in little endian, but the integers are in big endian? I doubt a hardware device would mix endianess in its output of binary data.
Yes 0xFFFF is -1 for a signed short.
Also, why wouldn't you use the BinaryReader class?
The value of FFFF for a "short" is -1 but the value of FFFF for an "unsigned short" is 65536.
In this case you should make sure you are using an unsigned short if you are sure that all of your values will be positive.
Related
Suppose I have an array that ends with the most signifiant bit of the most significant byte equaling 1.
My understanding is that if this is the case, BigInteger will treat this as a negative number, by design.
BigInteger numberToShorten = new BigInteger(toEncode);
if (numberToShorten.Sign == -1)
{
// problem with twos compliment or the last bit of the last byte is equal to 1
throw new Exception("Unexpected negative number");
}
To solve this problem, I think I need to add a dummy zero bit to the array, prior to converting the array. I can easily do this using Array.Resize().
My question is, how should I test if the last bit is indeed equal to 1?
I'm pretty weak on my boolean logic now, and am thinking I need to AND the values and test for equality, but not able to get the syntax correct in C#. Something like this:
byte temp = toEncode[toEncode.Length - 1];
if (temp == ???)
{
Array.Resize(ref toEncode, toEncode.Length +1);
}
If the number to be converted to BigInteger is always positive then you don't actually need to test at all; just appending a zero byte will always work correctly.
For completeness, checking if the MSB of any one byte is set is done with
if (byte & 0x80 == 0x80) ...
Found the answer at the bottom of this MSDN article (I think)
ulong originalNumber = UInt64.MaxValue;
byte[] bytes = BitConverter.GetBytes(originalNumber);
if (originalNumber > 0 && (bytes[bytes.Length - 1] & 0x80) > 0)
{
byte[] temp = new byte[bytes.Length];
Array.Copy(bytes, temp, bytes.Length);
bytes = new byte[temp.Length + 1];
Array.Copy(temp, bytes, temp.Length);
}
BigInteger newNumber = new BigInteger(bytes);
Console.WriteLine("Converted the UInt64 value {0:N0} to {1:N0}.",
originalNumber, newNumber);
// The example displays the following output:
// Converted the UInt64 value 18,446,744,073,709,551,615 to 18,446,744,073,709,551,615.
I'm not sure if this is necessary, have you tried it? In any case, assuming that temp contains a byte and you are trying to test whether that byte ends in 1, this is how you should do it:
if((temp & 0x01)==1)
{
...
}
That amounts to:
Temp: ???????z
0x01: 00000001
Temp & 0x01: 0000000z
Where z represents the bit you are trying to test, and you can now just test whether the result is equal to 1 or not.
If you are trying to test whether the byte starts with 1, do:
if((temp & 0x80)==0x80) //0x80 is 10000000 in binary
{
...
}
I'm trying to debug some bit shifting operations and I need to visualize the bits as they exist before and after a Bit-Shifting operation.
I read from this answer that I may need to handle backfill from the shifting, but I'm not sure what that means.
I think that by asking this question (how do I print the bits in a int) I can figure out what the backfill is, and perhaps some other questions I have.
Here is my sample code so far.
static string GetBits(int num)
{
StringBuilder sb = new StringBuilder();
uint bits = (uint)num;
while (bits!=0)
{
bits >>= 1;
isBitSet = // somehow do an | operation on the first bit.
// I'm unsure if it's possible to handle different data types here
// or if unsafe code and a PTR is needed
if (isBitSet)
sb.Append("1");
else
sb.Append("0");
}
}
Convert.ToString(56,2).PadLeft(8,'0') returns "00111000"
This is for a byte, works for int also, just increase the numbers
To test if the last bit is set you could use:
isBitSet = ((bits & 1) == 1);
But you should do so before shifting right (not after), otherwise you's missing the first bit:
isBitSet = ((bits & 1) == 1);
bits = bits >> 1;
But a better option would be to use the static methods of the BitConverter class to get the actual bytes used to represent the number in memory into a byte array. The advantage (or disadvantage depending on your needs) of this method is that this reflects the endianness of the machine running the code.
byte[] bytes = BitConverter.GetBytes(num);
int bitPos = 0;
while(bitPos < 8 * bytes.Length)
{
int byteIndex = bitPos / 8;
int offset = bitPos % 8;
bool isSet = (bytes[byteIndex] & (1 << offset)) != 0;
// isSet = [True] if the bit at bitPos is set, false otherwise
bitPos++;
}
From a library I'm working with I recieve an array of ushort.
I want to convert them in an array of float: The first ushort represents the 16 MSB of the first float and the second ushort is the 16 LSB of the first float, and so on.
I tried with something like the following, but the value is cast as the value of the integer, not the raw bits:
ushort[] buffer = { 0xBF80, 0x0000 };
float f = (uint)buffer[0] << 16 | buffer[1];
// expected result => f == -1 (0xBF800000)
// effective result => f == 3.21283686E+9 (0x4F3F8000)
Any suggestion?
Have a look at the System.BitConverter class.
In particular, the ToSingle method which takes a sequence of bytes and converts them to a float.
ushort[] buffer = {0xBF80, 0x0000};
byte[] bytes = new byte[4];
bytes[0] = (byte)(buffer[1] & 0xFF);
bytes[1] = (byte)(buffer[1] >> 8);
bytes[2] = (byte)(buffer[0] & 0xFF);
bytes[3] = (byte)(buffer[0] >> 8);
float value = BitConverter.ToSingle( bytes, 0 );
EDIT
In the example, I had reversed the MSB/LSB order.. Now it is correct
You should use the BitConverter class for that.
Convert the two ushorts to byte arrays with BitConverter.GetBytes(UInt16), concatenate the two arrays and use BitConverter.ToSingle(byte[] value,int startIndex) to convert the 4 bytes in the resulting array to a float.
I'd look at the System.BitConverter class. You can use BitConverter.GetBytes to turn your ushorts into byte arrays, then combine your byte arrays and use BitConverter to turn the byte array into a float.
Use a C# union:
[System.Runtime.InteropServices.StructLayout(LayoutKind.Explicit)]
public struct FloatUShortUnion {
[System.Runtime.InteropServices.FieldOffset(0)]
float floatValue;
[System.Runtime.InteropServices.FieldOffset(0)]
ushort short1;
[System.Runtime.InteropServices.FieldOffset(16)]
ushort short2;
}
You will need to use System.BitConverter, and convert the shorts to bytes.
http://msdn.microsoft.com/en-us/library/system.bitconverter.tosingle.aspx
I know this threads super old but I had a similar problem nothing above fixed. I had to send a decimal value to a plc using modbus and my only option to write to the registers was a ushort array. Luckily the software on the plc could convert the ushort array back to the decimal, the issue was reading the data from the plc. Here is what I was able to get to work. ConvertTo takes the users input and creates the ushort array for the write and ConvertFrom takes the ushort array the program receives from the plc and converts it back to the float.
private ushort[] ConvertTo(string value)
{
var bytes = BitConverter.GetBytes(float.Parse(value));
return new ushort[] {
BitConverter.ToUInt16(bytes, 0),
BitConverter.ToUInt16(bytes, 2)
};
}
private float ConvertFrom(ushort valueOne, ushort valueTwo)
{
byte[][] final = Array.ConvertAll(new ushort[] { valueOne, valueTwo }, delegate (ushort item) { return BitConverter.GetBytes(item); });
return BitConverter.ToSingle(new byte[4] { final[0][0], final[0][1], final[1][0], final[1][1] }, 0);
}
Here is a method -
using System;
class Program
{
static void Main(string[] args)
{
//
// Create an array of four bytes.
// ... Then convert it into an integer and unsigned integer.
//
byte[] array = new byte[4];
array[0] = 1; // Lowest
array[1] = 64;
array[2] = 0;
array[3] = 0; // Sign bit
//
// Use BitConverter to convert the bytes to an int and a uint.
// ... The int and uint can have different values if the sign bit differs.
//
int result1 = BitConverter.ToInt32(array, 0); // Start at first index
uint result2 = BitConverter.ToUInt32(array, 0); // First index
Console.WriteLine(result1);
Console.WriteLine(result2);
Console.ReadLine();
}
}
Output
16385
16385
I just want to know how this is happening?
The docs for BitConverter.ToInt32 actually have some pretty good examples. Assuming BitConverter.IsLittleEndian returns true, array[0] is the least significant byte, as you've shown... although array[3] isn't just the sign bit, it's the most significant byte which includes the sign bit (as bit 7) but the rest of the bits are for magnitude.
So in your case, the least significant byte is 1, and the next byte is 64 - so the result is:
( 1 * (1 << 0) ) + // Bottom 8 bits
(64 * (1 << 8) ) + // Next 8 bits, i.e. multiply by 256
( 0 * (1 << 16)) + // Next 8 bits, i.e. multiply by 65,536
( 0 * (1 << 24)) // Top 7 bits and sign bit, multiply by 16,777,216
which is 16385. If the sign bit were set, you'd need to consider the two cases differently, but in this case it's simple.
It converts like it was a number in base 256. So in your case : 1+64*256 = 16385
Looking at the .Net 4.0 Framework reference source, BitConverter does work how Jon's answer said, though it uses pointers (unsafe code) to work with the array.
However, if the second argument (i.e., startindex) is divisible by 4 (as is the case in your example), the framework takes a shortcut. It takes a byte pointer to the value[startindex], casts it to an int pointer, then dereferences it. This trick works regardless of whether IsLittleEndian is true.
From a high level, this basically just means the code is pointing at 4 bytes in the byte array and categorically declaring, "the chunk of memory over there is an int!" (and then returning a copy of it). This makes perfect sense when you take into account that under the hood, an int is just a chunk of memory.
Below is the source code of the framework ToUint32 method:
return (uint)ToInt32(value, startIndex);
array[0] = 1; // Lowest // 0x01 array[1] = 64; //
0x40 array[2] = 0; // 0x00 array[3] = 0; // Sign bit
0x00
If you combine each hex value 0x00004001
The MSDN documentatin explains everything
You can look for yourself - https://referencesource.microsoft.com/#mscorlib/system/bitconverter.cs,e8230d40857425ba
If the data is word-aligned, it will simply cast the memory pointer to an int32.
return *((int *) pbyte);
Otherwise, it uses bitwise logic from the byte memory pointer values.
For those of you who are having trouble with Little Endien and Big Endien. I use the following wrapper functions to take care of it.
public static Int16 ToInt16(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
{
return BitConverter.ToInt16(BitConverter.IsLittleEndian ? data.Skip(offset).Take(2).Reverse().ToArray() : data, 0);
}
return BitConverter.ToInt16(data, offset);
}
public static Int32 ToInt32(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
{
return BitConverter.ToInt32(BitConverter.IsLittleEndian ? data.Skip(offset).Take(4).Reverse().ToArray() : data, 0);
}
return BitConverter.ToInt32(data, offset);
}
public static Int64 ToInt64(byte[] data, int offset)
{
if (BitConverter.IsLittleEndian)
{
return BitConverter.ToInt64(BitConverter.IsLittleEndian ? data.Skip(offset).Take(8).Reverse().ToArray() : data, 0);
}
return BitConverter.ToInt64(data, offset);
}
I have this code for converting a byte[] to float[].
public float[] ConvertByteToFloat(byte[] array)
{
float[] floatArr = new float[array.Length / sizeof(float)];
int index = 0;
for (int i = 0; i < floatArr.Length; i++)
{
floatArr[i] = BitConverter.ToSingle(array, index);
index += sizeof(float);
}
return floatArr;
}
Problem is, I usually get a NaN result! Why should this be? I checked if there is data in the byte[] and the data seems to be fine. If it helps, an example of the values are:
new byte[] {
231,
255,
235,
255,
}
But this returns NaN (Not a Number) after conversion to float. What could be the problem? Are there other better ways of converting byte[] to float[]? I am sure that the values read into the buffer are correct since I compared it with my other program (which performs amplification for a .wav file).
If endianness is the problem, you should check the value of BitConverter.IsLittleEndian to determine if the bytes have to be reversed:
public static float[] ConvertByteToFloat(byte[] array) {
float[] floatArr = new float[array.Length / 4];
for (int i = 0; i < floatArr.Length; i++) {
if (BitConverter.IsLittleEndian) {
Array.Reverse(array, i * 4, 4);
}
floatArr[i] = BitConverter.ToSingle(array, i * 4);
}
return floatArr;
}
Isn't the problem that the 255 in the exponent represents NaN (see Wikipedia in the exponent section), so you should get a NaN. Try changing the last 255 to something else...
If it's the endianness that is wrong (you are reading big endian numbers) try these (be aware that they are "unsafe" so you have to check the unsafe flag in your project properties)
public static unsafe int ToInt32(byte[] value, int startIndex)
{
fixed (byte* numRef = &value[startIndex])
{
var num = (uint)((numRef[0] << 0x18) | (numRef[1] << 0x10) | (numRef[2] << 0x8) | numRef[3]);
return (int)num;
}
}
public static unsafe float ToSingle(byte[] value, int startIndex)
{
int val = ToInt32(value, startIndex);
return *(float*)&val;
}
I assume that your byte[] doesn't contain the binary representation of floats, but either a sequence of Int8s or Int16s. The examples you posted don't look like audio samples based on float (neither NaN nor -2.41E+24 are in the range -1 to 1. While some new audio formats might support floats outside that range, traditionally audio data consists of signed 8 or 16 bit integer samples.
Another thing you need to be aware of is that often the different channels are interleaved. For example it could contain the first sample for the left, then for the right channel, and then the second sample for both... So you need to separate the channels while parsing.
It's also possible, but uncommon that the samples are unsigned. In which case you need to remove the offset from the conversion functions.
So you first need to parse current position in the byte array into an Int8/16. And then convert that integer to a float in the range -1 to 1.
If the format is little endian you can use BitConverter. Another possibility that works with both endiannesses is getting two bytes manually and combining them with a bit-shift. I don't remember if little or big endian is common. So you need to try that yourself.
This can be done with functions like the following(I didn't test them):
float Int8ToFloat(Int8 i)
{
return ((i-Int8.MinValue)*(1f/0xFF))-0.5f;
}
float Int16ToFloat(Int16 i)
{
return ((i-Int16.MinValue)*(1f/0xFFFF))-0.5f;
}
Depends on how you want to convert the bytes to float. Start out by trying to pin-point what is actually stored in the byte array. Perhaps there is a file format specification? Other transformations in your code?
In the case each byte should be converted to a float between 0 and 255:
public float[] ConvertByteToFloat(byte[] array)
{
return array.Select(b => (float)b).ToArray();
}
If the bytes array contains binary representation of floats, there are several representation and if the representation stored in your file does not match the c# language standard floating point representation (IEEE 754) weird things like this will happen.