We Keep Coding

C# extract bit ranges from byte array

I need to extract some bit ranges from a 16-byte value, e.g.:
bit 0 = first thing
next 54 bits = second thing
next 52 bits = third thing
last 21 bits = fourth thing
.net doesn't have a UInt128 structure, well it has the BigInteger class, but I'm not sure that's right for the job, maybe it is?
I have found a third party library that can read bits from a stream, but when trying to convert them back to UInt64's using the BitConverter, it will fail, as 54 bits isn't long enough for a UInt64, but it's too long for a UInt32
My immediate thought was the bit shifting was the way to do this, but now I'm not so sure how to proceed, seeing as I can't think of a good way of handling the original 16 bytes.
Any suggestions or comments would be appreciated.

Here's some untested code. I'm sure that there are bugs in it (whenever I write code like this, I get shifts, masks, etc. wrong). However, it should be enough to get you started. If you get this working and there are only a few problems, let me know in the comments and I'll fix things. If you can't get it to work, let me know as well, and I'll delete the answer. If it requires a major rewrite, post your working code as an answer and let me know.
The other thing to worry about with this (since you mentioned that this comes from a file) is endian-ness. Not all computer architectures represent values in the same way. I'll leave any byte swizzling (if needed) to you.
First, structs in C++ are basically the same as classes (though people think they are different). In C#, they are very different. A struct in C# is a Value Type. When you do value type assignment, the compiler makes a copy of the value of the struct, rather than just making a copy to a reference to the object (like it does with classes). Value types have an implicit default constructor that initializes all members to their default (zero or null) values.
Marking the struct with [StructLayout(LayoutKind.Sequential)] tells the compiler to layout the members in the specified order (they compiler doesn't have to normally). This allows you to pass a reference to one of these (via P/Invoke) to a C program if you want to.
So, my struct starts off this way:
[StructLayout(LayoutKind.Sequential)]
public struct Struct128
{
//not using auto-properties with private setters on purpose.
//This should look like a single 128-bit value (in part, because of LayoutKind.Sequential)
private ulong _bottom64bits;
private ulong _top64bits;
}
Now I'm going to add members to that struct. Since you are getting the 128 bits from a file, don't try to read the data into a single 128-bit structure (if you can figure out how (look up serialization), you can, but...). Instead, read 64 bits at a time and use a constructor like this one:
public Struct128(ulong bottom64, ulong top64)
{
_top64bits = top64;
_bottom64bits = bottom64;
}
If you need to write the data in one of these back into the file, go get it 64-bits at a time using read-only properties like this:
//read access to the raw storage
public ulong Top64 => _top64bits;
public ulong Bottom64 => _bottom64bits;
Now we need to get and set the various bit-ish values out of our structure. Getting (and setting) the first thing is easy:
public bool FirstThing
{
get => (_bottom64bits & 0x01) == 1;
set
{
//set or clear the 0 bit
if (value)
{
_bottom64bits |= 1ul;
}
else
{
_bottom64bits &= (~1ul);
}
}
}
Getting/setting the second and fourth things are very similar. In both cases, to get the value, you mask away all but the important bits and then shift the result. To set the value, you take the property value, shift it to the right place, zero out the bits in the appropriate (top or bottom) value stored in the structure and OR in the new bits (that you set up by shifting)
//bits 1 through 55
private const ulong SecondThingMask = 0b111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1110;
public ulong SecondThing
{
get => (_bottom64bits & SecondThingMask) >> 1;
set
{
var shifted = (value << 1) & SecondThingMask;
_bottom64bits = (_bottom64bits & (~SecondThingMask)) | shifted;
}
}
and
//top 21 bits
private const ulong FourthThingMask = 0b1111_1111_1111_1111_1111_1000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000;
//to shift the top 21 bits down to the bottom 21 bits, need to shift 64-21
private const int FourthThingShift = 64 - 21;
public uint FourthThing
{
get => (uint)((_top64bits & FourthThingMask) >> FourthThingShift);
set
{
var shifted = ((ulong)value << FourthThingShift) & FourthThingMask;
_top64bits = (_top64bits & (~FourthThingMask)) | shifted;
}
}
It's the third thing that is tricky. To get the value, you need to mask the correct bits out of both the top and bottom values, shift them to the right positions and return the ORed result.
To set the value, you need to take the property value, split it into upper and lower portions and then do the same kind of magic ORing that was done for the second and fourth things:
//the third thing is the hard part.
//The bottom 55 bits of the _bottom64bits are dedicate to the 1st and 2nd things, so the next 9 are the bottom 9 of the 3rd thing
//The other 52-9 (=43) bits come-from/go-to the _top64bits
//top 9 bits
private const ulong ThirdThingBottomMask = 0b1111_1111_1000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000;
//bottom 43 bits
private const ulong ThirdThingTopMask = 0b111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111;
private const int ThirdThingBottomShift = 64 - 9;
//bottom 9 bits
private const ulong ThirdThingBottomSetMask = 0b1_1111_1111;
//all but the bottom 9 bits
private const ulong ThirdThingTopSetMask = 0b1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1110_0000_0000;
//52 bits total
private const ulong ThirdThingOverallMask = 0b1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111;
public ulong ThirdThing
{
get
{
var bottom = (_bottom64bits & ThirdThingBottomMask) >> ThirdThingBottomShift;
var top = (_top64bits & ThirdThingTopMask) << 9;
return top | bottom;
}
set
{
var masked = value & ThirdThingOverallMask;
var bottom = (masked & ThirdThingBottomSetMask) << ThirdThingBottomShift;
_bottom64bits = (_bottom64bits & (~ThirdThingBottomSetMask)) | bottom;
var top = (masked & ThirdThingTopSetMask) >> 9;
_top64bits = (_top64bits & (~ThirdThingTopSetMask)) | top;
}
}
I hope this is useful. Let me know.

Set a specific bit in an int

I need to mask certain string values read from a database by setting a specific bit in an int value for each possible database value. For example, if the database returns the string "value1" then the bit in position 0 will need to be set to 1, but if the database returns "value2" then the bit in position 1 will need to be set to 1 instead.
How can I ensure each bit of an int is set to 0 originally and then turn on just the specified bit?

If you have an int value "intValue" and you want to set a specific bit at position "bitPosition", do something like:
intValue = intValue | (1 << bitPosition);
or shorter:
intValue |= 1 << bitPosition;
If you want to reset a bit (i.e, set it to zero), you can do this:
intValue &= ~(1 << bitPosition);
(The operator ~ reverses each bit in a value, thus ~(1 << bitPosition) will result in an int where every bit is 1 except the bit at the given bitPosition.)

To set everything to 0, AND the value with 0x00000000:
int startValue = initialValue & 0x00000000;
//Or much easier :)
int startValue = 0;
To then set the bit, you have to determine what number has just that bit set and OR it. For example, to set the last bit:
int finalValue = startValue | 0x00000001;
As #Magus points out, to unset a bit you do the exact opposite:
int finalValue = startValue & 0xFFFFFFFE;
//Or
int finalValue = startValue & ~(0x00000001);
The ~ operatior is bitwise not which flips every bit.

so, this?
enum ConditionEnum
{
Aaa = 0,
Bbb = 1,
Ccc = 2,
}
static void SetBit(ref int bitFlag, ConditionEnum condition, bool bitValue)
{
int mask = 1 << (int)condition;
if (bitValue)
bitFlag |= mask;
else
bitFlag &= ~mask;
}

Just provide a value, bit value and position. Note that you might be able to modify this to work for other types.
public static void SetBit(ref int value, bool bitval, int bitpos)
{
if (!bitval) value&=~(1<<bitpos); else value|=1<<bitpos;
}

not being able to convert from FILETIME (windows time) to dateTime ( I get a different date )

Most of the files I read get the right time when using the following method to convert:
// works great most of the time
private static DateTime convertToDateTime(System.Runtime.InteropServices.ComTypes.FILETIME time)
{
long highBits = time.dwHighDateTime;
highBits = highBits << 32;
return DateTime.FromFileTimeUtc(highBits + time.dwLowDateTime);
}
Here I have an example in visual studio to show how this method sometimes does not work for example I will show the actual file in my computer and the debug. So the file that happens to be in my debug is:
"A:\Users\Tono\Documents\Visual Studio 2010\Projects\WpfApplication4\WpfApplication4\obj\x86\Debug\App.g.cs"
And here is the FILETIME that I am trying to convert to DateTime "I need the LastWriteTime by the way"
Here you can see that dwHighDateTime = 30136437 and also that dwLowDateTime = -2138979250 from that file.
And when I run my method plus other techniques I get the following dates:
So so far everything seems to be working great. But why is that that when I browse and look for that specific file in windows I get a different date !? Here is the date that I get when seeing the file's properties:
Why does the dates don't match? What am I doing wrong?

You need to combine the LS and MS values bitwise, not arithmetically.
Try:
ulong high = 30136437;
unchecked
{
int low = -2138979250;
uint uLow = (uint)low;
high = high << 32;
Date dt = DateTime.FromFileTime((long) (high | (ulong)uLow));
}
Or any of the following should work too:
long highBits = time.dwHighDateTime;
highBits = highBits << 32;
return DateTime.FromFileTimeUtc(highBits + (long) (uint) time.dwLowDateTime);
return DateTime.FromFileTimeUtc(highBits | (long) (uint) time.dwLowDateTime);
return DateTime.FromFileTimeUtc(highBits + ((long)low & 0xFFFFFFFF))
return DateTime.FromFileTimeUtc(highBits | ((long)low & 0xFFFFFFFF))
You can get away with adding rather than a bitwise-or if you are sure the values are positive (and have no bits in common). But bitwise-or expresses the intent better.

I'm a bit late to the party, but this has worked reliably for me:
public static class FILETIMEExtensions
{
public static DateTime ToDateTime(this System.Runtime.InteropServices.ComTypes.FILETIME time)
{
ulong high = (ulong)time.dwHighDateTime;
uint low = (uint)time.dwLowDateTime;
long fileTime = (long)((high << 32) + low);
try
{
return DateTime.FromFileTimeUtc(fileTime);
}
catch
{
return DateTime.FromFileTimeUtc(0xFFFFFFFF);
}
}
}
Note: Don't trust Windows Explorer. Use File.GetLastWriteTimeUtc method, for example, to verify what the file system actually has against what this extension method returns. Explorer has some bugs in it that don't update file times in certain situations. Cheers! :)
Note: To test this, you need to use maximum values. So, assuming dwHighDateTime = dwLowDateTime = UInt32.MaxValue = 4294967295 = 0xFFFFFFFF, it follows that (long)(((ulong)UInt32.MaxValue << 32) + UInt32.MaxValue) = -1 = 0xFFFFFFFFFFFFFFFF. Unfortunately, the fallacy in the Windows API seems to be that eventually the time needs to be casted to a long value in order to work with it for any useful applications (since most Windows API methods take the file time as a long value), which means once the leading bit is high (1) on dwHighDateTime, the value becomes negative. Lets try with the maximum time not being high. Assuming dwHighDateTime = Int32.MaxValue = 2147483647 = 0x7FFFFFFF and dwLowDateTime = UInt32.MaxValue = 4294967295 = 0xFFFFFFFF, it follows that (long)(((ulong)Int32.MaxValue << 32) + UInt32.MaxValue) = 0x7FFFFFFFFFFFFFFF.
Note: 0x7FFFFFFFFFFFFFFF is already much larger than DateTime.MaxValue.ToFileTimeUtc() = 2650467743999999999 = 0x24C85A5ED1C04000, rendering numbers that large already useless for any practical applications in .NET.

This is another method that I have seen to convert a FileTime structure to a long (using a coded operator in the struct), which can then easily be converted to DateTime using the DateTime.FromFileTime functions:
public struct FileTime
{
public uint dwLowDateTime;
public uint dwHighDateTime;
public static implicit operator long(FileTime fileTime)
{
long returnedLong;
// Convert 4 high-order bytes to a byte array
byte[] highBytes = BitConverter.GetBytes(fileTime.dwHighDateTime);
// Resize the array to 8 bytes (for a Long)
Array.Resize(ref highBytes, 8);
// Assign high-order bytes to first 4 bytes of Long
returnedLong = BitConverter.ToInt64(highBytes, 0);
// Shift high-order bytes into position
returnedLong = returnedLong << 32;
// Or with low-order bytes
returnedLong = returnedLong | fileTime.dwLowDateTime;
// Return long
return returnedLong;
}
}

I have tried the following and non of them get me the right time:
And I got the method from here

dwLowDateTime and dwHighDateTime should be uint and it looks like they are int. Changing this will most likely fix it though as #Joe pointed out you should still use | instead of +.

C# get and set the high order word of an integer

What's an efficient or syntactically simple way to get and set the high order part of an integer?

There are multiple ways of achieving that, here are some of them.
Using the Bitwise and/or Shift operators
Applying a right shift in an integer will move the bits to the right, putting zeros to the left.
In the case below, it will shift the size of a short (Int16, as 16 bits).
Applying a logical AND (&) operation in an integer like 0x0000FFFF will basically 'cut' the value (where it's F) and ignore the rest (where it's 0).
Remember that in the end it's just a 0b_1 AND 0b_1 = 0b_1 operation, so any 0b_0 AND 0b_1 will result in 0b_0.
Applying a logical OR (|) operation will basically merge the two numbers in this case, like 0b_10 | 0b_01 = 0b_11.
Code:
uint number = 0xDEADBEEF;
//Get the higher order value.
var high = number >> 16;
Console.WriteLine($"High: {high:X}");
//Get the lower order value.
var low = number & 0xFFFF; //Or use 0x0000FFFF
Console.WriteLine($"Low: {low:X}");
//Set a high order value (you can also use 0xFFFF instead of 0x0000FFFF).
uint newHigh = 0xFADE;
number = number & 0x0000FFFF | newHigh << 16;
Console.WriteLine($"New high: {number:X}");
//Set a low order value.
uint newLow = 0xC0DE;
number = number & 0xFFFF0000 | newLow & 0x0000FFFF;
Console.WriteLine($"New low: {number:X}");
Output:
High: DEAD
Low: BEEF
New high: FADEBEEF
New low: FADEC0DE
Using FieldOffsetAttribute in a struct
C# has excellent support for variables sharing the same memory location, and bits structuring.
Since C# has no macro functions like in C, you can use the union approach to speed things up. It's more performant than passing the variable to methods or extension methods.
You can do that by simply creating a struct with explicit layout and setting the offset of the fields:
Code:
using System;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Explicit)]
struct WordUnion
{
[FieldOffset(0)]
public uint Number;
[FieldOffset(0)]
public ushort Low;
[FieldOffset(2)]
public ushort High;
}
public class MainClass
{
public static void Main(string[] args)
{
var x = new WordUnion { Number = 0xABADF00D };
Console.WriteLine("{0:X} {1:X} {2:X}", x.Number, x.High, x.Low);
x.Low = 0xFACE;
Console.WriteLine("{0:X} {1:X} {2:X}", x.Number, x.High, x.Low);
x.High = 0xDEAD;
Console.WriteLine("{0:X} {1:X} {2:X}", x.Number, x.High, x.Low);
}
}
Output:
ABADF00D ABAD F00D
ABADFACE ABAD FACE
DEADFACE DEAD FACE
Mind that with Visual Studio 2029 (16.7), you still may get zeros in x.High and x.Low when adding the variable x inside the Watch or by hovering your cursor on top of the variables x.High and x.Low directly.
Using unsafe and pointer element access operator []
To a more akin to C programming, but in C#, use unsafe:
Code:
unsafe
{
uint value = 0xCAFEFEED;
// x86 is using low-endian.
// So low order array number gets the low order of the value
// And high order array number gets the high order of the value
Console.WriteLine("Get low order of {0:X}: {1:X}",
value, ((ushort*) &value)[0]);
Console.WriteLine("Get high order of {0:X}: {1:X}",
value, ((ushort*) &value)[1]);
((ushort*) &value)[1] = 0xABAD;
Console.WriteLine("Set high order to ABAD: {0:X}", value);
((ushort*) &value)[0] = 0xFACE;
Console.WriteLine("Set low order to FACE: {0:X}", value);
}
Output:
Get low order of CAFEFEED: FEED
Get high order of CAFEFEED: CAFE
Set high order to ABAD: ABADFEED
Set low order to FACE: ABADFACE
Using unsafe and pointer member access operator ->
Another unsafe approach, but this time accessing a member from the WordUnion struct declared in a previous example:
Code:
unsafe
{
uint value = 0xCAFEFEED;
Console.WriteLine("Get low order of {0:X}: {1:X}",
value, ((WordUnion*) &value)->Low);
Console.WriteLine("Get high order of {0:X}: {1:X}",
value, ((WordUnion*) &value)->High);
((WordUnion*) &value)->High = 0xABAD;
Console.WriteLine($"Set high order to ABAD: {value:X}");
((WordUnion*) &value)->Low = 0xFACE;
Console.WriteLine($"Set low order to FACE: {value:X}");
}
Output:
Get low order of CAFEFEED: FEED
Get high order of CAFEFEED: CAFE
Set high order to ABAD: ABADFEED
Set low order to FACE: ABADFACE
Using the BitConverter class
It simply gets 16 bits (2 bytes, a short/Int16) from the specified number. The offset can be controlled by the second parameter.
Code:
uint value = 0xCAFEFEED;
var low = BitConverter.ToInt16(BitConverter.GetBytes(value), 0);
var high = BitConverter.ToInt16(BitConverter.GetBytes(value), 2);
Console.WriteLine($"Low: {low:X}");
Console.WriteLine($"High: {high:X}");
Output:
Low: 0xCAFE
High: 0xFEED

It's the same as in C/C++:
// get the high order 16 bits
int high = 0x12345678 >> 16; // high = 0x1234
// set the high order 16 bits
high = (high & 0x0000FFFF) + (0x5678 << 16); // high = 0x56781234
EDIT: Because I'm in a good mood, here you go. Just remember, immutable types are immutable! The 'set' functions need to be assigned to something.
public static class ExtensionMethods
{
public int LowWord(this int number)
{ return number & 0x0000FFFF; }
public int LowWord(this int number, int newValue)
{ return (number & 0xFFFF0000) + (newValue & 0x0000FFFF); }
public int HighWord(this int number)
{ return number & 0xFFFF0000; }
public int HighWord(this int number, int newValue)
{ return (number & 0x0000FFFF) + (newValue << 16); }
}
EDIT 2: On second thoughts, if you really need to do this and don't want the syntax everywhere, use Michael's solution. +1 to him for showing me something new.

I guess you don't want calculations when you want the Hiword / Hibyte or the LoWord / Lobyte,
if a System.Int32 starts at address 100 (so it occupies address 100 to 103), you want as a LoWord the two bytes starting at address 100 and 101 and Hiword is address 102 and 103.
This can be achieved using the class BitConverter. This class doesn't do anything with the bits, it only uses the addresses to return the requested value.
As the size of types like int / long are different per platform, and WORD and DWORD are a bit confusing, I use the System types System.Int16/Int32/Int64. No one will ever have any problems guessing the number of bits in a System.Int32.
With BitConverter you can convert any integer to the array of bytes starting on that location and convert an array of bytes of the proper length to the corresponding integer. No calculations needed and no bits will change,
Suppose you have a System.Int32 X (which is a DWORD in old terms)
LOWORD: System.Int16 y = BitConverter.ToInt16(BitConverter.GetBytes(x), 0);
HIWORD: System.Int16 y = BitConverter.ToInt16(BitConverter.GetBytes(x), 2);
The nice thing is that this works with all lengths, you don't have to combine functions like LOBYTE and HIWORD to get the third byte:
HIByte(Hiword(x)) will be like: BitConverter.GetBytes(x)[3]

Another Alternative
public class Macro
{
public static short MAKEWORD(byte a, byte b)
{
return ((short)(((byte)(a & 0xff)) | ((short)((byte)(b & 0xff))) << 8));
}
public static byte LOBYTE(short a)
{
return ((byte)(a & 0xff));
}
public static byte HIBYTE(short a)
{
return ((byte)(a >> 8));
}
public static int MAKELONG(short a, short b)
{
return (((int)(a & 0xffff)) | (((int)(b & 0xffff)) << 16));
}
public static short HIWORD(int a)
{
return ((short)(a >> 16));
}
public static short LOWORD(int a)
{
return ((short)(a & 0xffff));
}
}

I use these 2 function...
public static int GetHighint(long intValue)
{
return Convert.ToInt32(intValue >> 32);
}
public static int GetLowint(long intValue)
{
long tmp = intValue << 32;
return Convert.ToInt32(tmp >> 32);
}

Is there a way to perform a circular bit shift in C#?

I know that the following is true
int i = 17; //binary 10001
int j = i << 1; //decimal 34, binary 100010
But, if you shift too far, the bits fall off the end. Where this happens is a matter of the size of integer you are working with.
Is there a way to perform a shift so that the bits rotate around to the other side? I'm looking for a single operation, not a for loop.

If you know the size of type, you could do something like:
uint i = 17;
uint j = i << 1 | i >> 31;
... which would perform a circular shift of a 32 bit value.
As a generalization to circular shift left n bits, on a b bit variable:
/*some unsigned numeric type*/ input = 17;
var result = input << n | input >> (b - n);
#The comment, it appears that C# does treat the high bit of signed values differently. I found some info on this here. I also changed the example to use a uint.

One year ago I've to implement MD4 for my undergraduate thesis. Here it is my implementation of circular bit shift using a UInt32.
private UInt32 RotateLeft(UInt32 x, Byte n)
{
return UInt32((x << n) | (x >> (32 - n)));
}

Sincce .NET Core 3.0 and up there's BitOperations.RotateLeft() and BitOperations.RotateRight() so you can just use something like
BitOperations.RotateRight(12, 3);
BitOperations.RotateLeft(34L, 5);
In previous versions you can use BitRotator.RotateLeft() and BitRotator.RotateRight() in Microsoft.VisualStudio.Utilities

Just as reference on how to do it, these two functions work perfectly for rotating the bits of 1/2word:
static public uint ShiftRight(uint z_value, int z_shift)
{
return ((z_value >> z_shift) | (z_value << (16 - z_shift))) & 0x0000FFFF;
}
static public uint ShiftLeft(uint z_value, int z_shift)
{
return ((z_value << z_shift) | (z_value >> (16 - z_shift))) & 0x0000FFFF;
}
It would be easy to extend it for any given size.

The extension methods for rotating bits of a uint (32 bits):
public static uint ROR(this uint x, int nbitsShift)
=> (x >> nbitsShift) | (x << (32 - nbitsShift));
public static uint ROL(this uint x, int nbitsShift)
=> (x << nbitsShift) | (x >> (32 - nbitsShift));