We Keep Coding

Why does BitConverter seemingly return incorrect results when converting floats and bytes?

I'm working in C# and attempting to pack four bytes into a float (the context is game development, where an RGBA color is packed into a single value). To do this, I'm using BitConverter, but certain conversions seem to result in incorrect bytes. Take the following example (using bytes 0, 0, 129, 255):
var before = new [] { (byte)0, (byte)0, (byte)129, (byte)255 };
var f = BitConverter.ToSingle(before, 0); // Results in NaN
var after = BitConverter.GetBytes(f); // Results in bytes 0, 0, 193, 255
Using https://www.h-schmidt.net/FloatConverter/IEEE754.html, I verified that the four bytes I started with (0, 0, 129, 255, equivalent to binary 00000000000000001000000111111111) represents the floating-point value 4.66338115943e-41. By flipping the endianness (binary 11111111100000010000000000000000), I get NaN (which matches f in the code above). But when I convert that float back to bytes, I get 0, 0, 193, 255 (note 193 when I'm expecting 129).
Curiously, running this same example with bytes 0, 0, 128, 255 is correct (the floating-point value f becomes -Infinity, then converting back to bytes yields 0, 0, 128, 255 again). Given this fact, I suspect NaN is relevant.
Can anyone shed some light on what's happening here?
Update: the question Converting 2 bytes to Short in C# was listed as a duplicate, but that's inaccurate. That question is attempting to convert bytes to a value (in that case, two bytes to a short) and incorrect endianness was giving an unexpected value. In my case, the actual float value is irrelevant (since I'm not using the converted value as a float). Instead, I'm attempting to effectively reinterpret four bytes as a float directly by first converting to a float, then converting back. As shown, that back-and-forth sometimes returns different bytes than the ones I sent in.
Second update: I'll simply my question. As Peter Duniho comments, BitConverter will never modify the bytes you pass in, but simply copy them to a new memory location and reinterpret the result. However, as my example shows, it is possible to send in four bytes (0, 0, 129, 255) which are internally copied and reinterpreted to a float, then convert that float back to bytes that are different than the originals (0, 0, 193, 255).
Endianness is frequently mentioned in relation to BitConverter. However, in this case, I feel endianness isn't the root issue. When I call BitConverter.ToSingle, I pass in an array of four bytes. Those bytes represent some binary (32 bits) which is converted to a float. By changing the endianness prior to the function call, all I'm doing is changing the bits I send into the function. Regardless of the value of those bits, it should be possible to convert them to a float (also 32 bits), then convert the float back to get the same bits I sent in. As demonstrated in my example, using bytes 0, 0, 129, 255 (binary 00000000000000001000000111111111) results in a floating-point value. I'd like to take that value (the float represented by those bits) and convert it to the original four bytes.
Is this possible in C# in all cases?

After research, experimentation, and discussion with friends, the root cause of this behavior (bytes changing when converted to and from a float) seems to be signaling vs. quiet NaNs (as Hans Passant also pointed out in a comment). I'm no expert on signaling and quiet NaNs, but from what I understand, quiet NaNs have the highest-order bit of the mantissa set to one, while signaling NaNs have that bit set to zero. See the following image (taken from https://www.h-schmidt.net/FloatConverter/IEEE754.html) for reference. I've drawn four colored boxes around each group of eight bits, as well as an arrow pointing to the highest-order mantissa bit.
Of course, the question I posted wasn't about floating-point bit layout or signaling vs. quiet NaNs, but simply asking why my encoded bytes were seemingly modified. The answer is that the C# runtime (or at least I assume it's the C# runtime) internally converts all signaling NaNs to quiet, meaning that the byte encoded at that position has its second bit swapped from zero to one.
For example, the bytes 0, 0, 129, 255 (encoded in the reverse order, I think due to endianness) puts the value 129 in the second byte (the green box). 129 in binary is 10000001, so flipping its second bit gives 11000001, which is 193 (exactly what I saw in my original example). This same pattern (the encoded byte having its value changed) applies to all bytes in the range 129-191 inclusive. Bytes 128 and lower aren't NaNs, while bytes 192 and higher are NaNs, but don't have their value modified because their second bit (placed at the highest-order mantissa bit) is already one.
So that answers why this behavior occurs, but in my mind, there are two questions remaining:
Is it possible to disable this behavior (converting signaling NaNs to quiet) in C#?
If not, what's the workaround?
The answer to the first question seems to be no (I'll amend this answer if I learn otherwise). However, it's important to note that this behavior doesn't appear consistent across all .NET versions. On my computer, NaNs are converted (i.e. my encoded bytes changed) on every .NET Framework version I tried (starting with 4.8.0, then working back down). NaNs appear to not be converted (i.e. my encoded bytes did not change) in .NET Core 3 and .NET 5 (I didn't test every available version). In addition, a friend was able to run the same sample code on .NET Framework 4.7.2, and surprisingly, the bytes were not modified on his machine. The internals of different C# runtimes isn't my area of expertise, but suffice to say there's variance among versions and computers.
The answer to the second question is to, as others have suggested, simply avoid the float conversion entirely. Instead, each set of four bytes (representing RGBA colors in my case) can either be encoded in an integer or added to a byte array directly.

What exactly is byte[] in C#?

In C#, byte is the data type for 8-bit unsigned integers, so a byte[] should be an array of integers who are between 0 and 255, just like an char[] is an array of characters.
But most of time when I encounter byte[], I see byte[] is used as a contiguous chunk of memory for storing raw representation of data.
How do these two relate to each other?
thanks

Well, a byte as datatype is exactly what you already said, an unsigned integer between 0 and 255. Furthermore this type needs exactly - believe it or not - one byte in your memory, thus also the name. This is why most readers that read byte per byte store those information in a structure that fits exactly the size of a byte - the byte-datatype.

Converting byte array to hexadecimal value using BitConverter class in c#?

I'm trying to convert a byte array into hexadecimal value using Bitconverter class.
long hexValue = 0X780B13436587;
byte[] byteArray = BitConverter.GetBytes ( hexValue );
string hexResult = BitConverter.ToString ( byteArray );
now if I execute the above code line by line, this is what I see
I thought hexResult string would be same as hexValue (i.e. 780B13436587h) but what I get is different, am I missing something, correct me if I'm wrong.
Thanks!

Endianness.
BitConverter uses CPU-endianness, which for most people means: little-endian. When humans write numbers, we tend to write big-endian (broadly speaking: you write the thousands, then hundreds, then tens, then the digits). For a CPU, big-endian means that the most-significant byte is first and the least-significant byte is last. However, unless you're using an Itanium, your CPU is probably little-endian, which means that the most-significant byte is last, and the least-significant byte is first. The CPU is implemented such that this doesn't matter unless you are peeking inside raw memory - it will ensure that numeric and binary arithmetic still works the way you expect. However, BitConverter works by peeking inside raw memory - hence you see the reversed data.
If you want the value in big-endian format, then you'll need to:
do it manually in big-endian order
check the BitConverter.IsLittleEndian value, and if true:
either reverse the input bytes
or reverse the output

If you look closely, the bytes in the output from BitConverter are reversed.
To get the hex-string for a number, you use the Convert class:
Convert.ToString(hexValue, 16);

It is the same number but reversed.
BitConverter.ToString can return string representation in reversed order:
http://msdn.microsoft.com/en-us/library/3a733s97(v=vs.110).aspx
"All the elements of value are converted. The order of hexadecimal strings returned by the ToString method depends on whether the computer architecture is little-endian or big-endian."

Converting an amount to a 4 byte array

I haven't ever had to deal with this before. I need to convert a sale amount (48.58) to a 4 byte array and use network byte order. The code below is how I am doing it, but it is wrong and I am not understanding why. Can anyone help?
float saleamount = 48.58F;
byte[] data2 = BitConverter.GetBytes(saleamount).Reverse().ToArray();
What I am getting is 66 66 81 236 in the array. I am not certain what it should be though. I am interfacing with a credit card terminal and need to send the amount in "4 bytes, fixed length, max value is 0xffffffff, use network byte order"

The first question you should ask is, "What data type?" IEEE single-precision float? Twos-complement integer? It's an integer, what is the implied scale? Is $48.53 represented as 4,853 or 485,300?
It's not uncommon for monetary values to be represented by an integer with an implied scale of either +2 or +4. In your example, $48.58 would be represented as the integer value 4858 or 0x000012FA.
Once you've established what they actually want...use an endian-aware BitConverter or BinaryWriter to create it. Jon Skeet's MiscUtil, for instance offers:
EndianBinaryReader
EndianBinaryWriter
BigEndianBitConverter
LittleEndianBitConverter
There are other implementations out there as well. See my answer to the question "Helpful byte array extensions to handle BigEndian data" for links to some.
Code you don't write it code you don't have to maintain.

Network byte order pseudo-synonym of big-endian, hence (as itsme86 mentioned already) so you can check BitConverter.IsLittleEndian:
float saleamount = 48.58F;
byte[] data2 = BitConverter.IsLittleEndian
? BitConverter.GetBytes(saleamount).Reverse().ToArray()
: BitConverter.GetBytes(saleamount);
But if you don't know this, probably you already using some protocol, which handle it.

How do I convert a int to an array of byte's and then back?

I need to send an integer through a NetworkStream. The problem is that I only can send bytes.
Thats why I need to split the integer in four byte's and send those and at the other end convert it back to a int.
For now I need this only in C#. But for the final project I will need to convert the four bytes to an int in Lua.
[EDIT]
How about in Lua?

BitConverter is the easiest way, but if you want to control the order of the bytes you can do bit shifting yourself.
int foo = int.MaxValue;
byte lolo = (byte)(foo & 0xff);
byte hilo = (byte)((foo >> 8) & 0xff);
byte lohi = (byte)((foo >> 16) & 0xff);
byte hihi = (byte)(foo >> 24);
Also.. the implementation of BitConverter uses unsafe and pointers, but it's short and simple.
public static unsafe byte[] GetBytes(int value)
{
byte[] buffer = new byte[4];
fixed (byte* numRef = buffer)
{
*((int*) numRef) = value;
}
return buffer;
}

Try
BitConverter.GetBytes()
http://msdn.microsoft.com/en-us/library/system.bitconverter.aspx
Just keep in mind that the order of the bytes in returned array depends on the endianness of your system.
EDIT:
As for the Lua part, I don't know how to convert back. You could always multiply by 16 to get the same functionality of a bitwise shift by 4. It's not pretty and I would imagine there is some library or something that implements it. Again, the order to add the bytes in depends on the endianness, so you might want to read up on that
Maybe you can convert back in C#?

For Lua, check out Roberto's struct library. (Roberto is one of the authors of Lua.) It is more general than needed for the specific case in question, but it isn't unlikely that the need to interchange an int is shortly followed by the need to interchange other simple types or larger structures.
Assuming native byte order is acceptable at both ends (which is likely a bad assumption, incidentally) then you can convert a number to a 4-byte integer with:
buffer = struct.pack("l", value)
and back again with:
value = struct.unpack("l", buffer)
In both cases, buffer is a Lua string containing the bytes. If you need to access the individual byte values from Lua, string.byte is your friend.
To specify the byte order of the packed data, change the format from "l" to "<l" for little-endian or ">l" for big-endian.
The struct module is implemented in C, and must be compiled to a DLL or equivalent for your platform before it can be used by Lua. That said, it is included in the Lua for Windows batteries-included installation package that is a popular way to install Lua on Windows systems.

Here are some functions in Lua for converting a 32-bit two's complement number into bytes and converting four bytes into a 32-bit two's complement number. A lot more checking could/should be done to verify that the incoming parameters are valid.
-- convert a 32-bit two's complement integer into a four bytes (network order)
function int_to_bytes(n)
if n > 2147483647 then error(n.." is too large",2) end
if n < -2147483648 then error(n.." is too small",2) end
-- adjust for 2's complement
n = (n < 0) and (4294967296 + n) or n
return (math.modf(n/16777216))%256, (math.modf(n/65536))%256, (math.modf(n/256))%256, n%256
end
-- convert bytes (network order) to a 32-bit two's complement integer
function bytes_to_int(b1, b2, b3, b4)
if not b4 then error("need four bytes to convert to int",2) end
local n = b1*16777216 + b2*65536 + b3*256 + b4
n = (n > 2147483647) and (n - 4294967296) or n
return n
end
print(int_to_bytes(256)) --> 0 0 1 0
print(int_to_bytes(-10)) --> 255 255 255 246
print(bytes_to_int(255,255,255,246)) --> -10

investigate the BinaryWriter/BinaryReader classes

Convert an int to a byte array and display : BitConverter ...
www.java2s.com/Tutorial/CSharp/0280__Development/Convertaninttoabytearrayanddisplay.htm
Integer to Byte - Visual Basic .NET answers
http://bytes.com/topic/visual-basic-net/answers/349731-integer-byte
How to: Convert a byte Array to an int (C# Programming Guide)
http://msdn.microsoft.com/en-us/library/bb384066.aspx

As Nubsis says, BitConverter is appropriate but has no guaranteed endianness.
I have an EndianBitConverter class in MiscUtil which allows you to specify the endianness. Of course, if you only want to do this for a single data type (int) you could just write the code by hand.
BinaryWriter is another option, and this does guarantee little endianness. (Again, MiscUtil has an EndianBinaryWriter if you want other options.)

To convert to a byte[]:
BitConverter.GetBytes(int)
http://msdn.microsoft.com/en-us/library/system.bitconverter.aspx
To convert back to an int:
BitConverter.ToInt32(byteArray, offset)
http://msdn.microsoft.com/en-us/library/system.bitconverter.toint32.aspx
I'm not sure about Lua though.
If you are concerned about endianness use John Skeet's EndianBitConverter. I've used it and it works seamlessly.
C# supports their own implementation of htons and ntohs as:
system.net.ipaddress.hosttonetworkorder()
system.net.ipaddress.networktohostorder()
But they only work on signed int16, int32, int64 which means you'll probably end up doing a lot of unnecessary casting to make them work, and if you're using the highest order bit for anything other than signing the integer, you're screwed. Been there, done that. ::tsk:: ::tsk:: Microsoft for not providing better endianness conversion support in .NET.