I have a bit field consisting of 64 bits:
long bitfield = 0;
I can set the bit for a given index as follows:
void Set(long index)
{
bitfield |= 1L << (int)(index % 64);
}
i.e. if the index is 0, 64, 128, ... then bit 0 is set, if the index is 1, 65, 129, ... then bit 1 is set, and so on.
Question: given an index and a count (or a lower and upper index), how can I set the bits for all indexes in this range without using a loop?
long SetRangeMask(int lower, int upper) // 3..7
{
if (! (lower <= upper)) throw new ArgumentException("...");
int size = upper - lower + 1; // 7 - 3 + 1 = 5
if (size >= 64) return -1;
long mask = (1 << size) - 1; // #00100000 - 1 = #000011111
return mask << lower | mask >> -lower; // #00011111 << 3 = #011111000
}
You could use a lookup table for combined bit masks
A real simple approach with no thought to special cases or optimizations like these questions raised, would look like:
static readonly private long[] maskLUT = new long[64,64] { /* generated */ };
void SetRange(long lobit, long hibit)
{
lobit %= 64;
hibit %= 64;
bitfield |= lobit<hibit? maskLUT[lobit,hibit] : maskLUT[hibit,lobit];
}
Thoughts:
you might consider an optimization that given [lobit...hibit], if hibit-lobit>=64 you can set all bits at once.
There is a bit of thought to be put in the connected-ness of regions given the fact that both boundaries can wrap around (do you wrap-around both boundaries first, or do you wraparound lobit, and use the delta to find the hibit from the wrapped boundary, like with the optimization mentioned before?)
You can use 2x-1 to create a mask x bits long, then shift it and OR it in, like so:
void Set( int index, int count ) {
bitfield |= (long)(Math.Pow( 2, count ) - 1) << ((index-count) % 64);
}
Update: I like to think that Math.Pow optimizes powers of two to a left shift, but it may not. If that's the case, you can get a little more performance by replacing the call to Math.Pow with the corresponding left shift:
public void Set( int index, int count ) {
bitfield |= ((2 << count - 1) - 1) << ((index-count) % 64);
}
Related
I have integer array and I need to convert it to byte array
but I need to take (only and just only) first 11 bit of each element of the هinteger array
and then convert it to a byte array
I tried this code
// ***********convert integer values to byte values
//***********to avoid the left zero padding on the byte array
// *********** first step : convert to binary string
// ***********second step : convert binary string to byte array
// *********** first step
string ByteString = Convert.ToString(IntArray[0], 2).PadLeft(11,'0');
for (int i = 1; i < IntArray.Length; i++)
ByteString = ByteString + Convert.ToString(IntArray[i], 2).PadLeft(11, '0');
// ***********second step
int numOfBytes = ByteString.Length / 8;
byte[] bytes = new byte[numOfBytes];
for (int i = 0; i < numOfBytes; ++i)
{
bytes[i] = Convert.ToByte(ByteString.Substring(8 * i, 8), 2);
}
But it takes too long time (if the file size large , the code takes more than 1 minute)
I need a very very fast code (very few milliseconds only )
can any one help me ?
Basically, you're going to be doing a lot of shifting and masking. The exact nature of that depends on the layout you want. If we assume that we pack little-endian from each int, appending on the left, so two 11-bit integers with positions:
abcdefghijk lmnopqrstuv
become the 8-bit chunks:
defghijk rstuvabc 00lmnopq
(i.e. take the lowest 8 bits of the first integer, which leaves 3 left over, so pack those into the low 3 bits of the next byte, then take the lowest 5 bits of the second integer, then finally the remaining 6 bits, padding with zero), then something like this should work:
using System;
using System.Linq;
static class Program
{
static string AsBinary(int val) => Convert.ToString(val, 2).PadLeft(11, '0');
static string AsBinary(byte val) => Convert.ToString(val, 2).PadLeft(8, '0');
static void Main()
{
int[] source = new int[1432];
var rand = new Random(123456);
for (int i = 0; i < source.Length; i++)
source[i] = rand.Next(0, 2047); // 11 bits
// Console.WriteLine(string.Join(" ", source.Take(5).Select(AsBinary)));
var raw = Encode(source);
// Console.WriteLine(string.Join(" ", raw.Take(6).Select(AsBinary)));
var clone = Decode(raw);
// now prove that it worked OK
if (source.Length != clone.Length)
{
Console.WriteLine($"Length: {source.Length} vs {clone.Length}");
}
else
{
int failCount = 0;
for (int i = 0; i < source.Length; i++)
{
if (source[i] != clone[i] && failCount++ == 0)
{
Console.WriteLine($"{i}: {source[i]} vs {clone[i]}");
}
}
Console.WriteLine($"Errors: {failCount}");
}
}
static byte[] Encode(int[] source)
{
long bits = source.Length * 11;
int len = (int)(bits / 8);
if ((bits % 8) != 0) len++;
byte[] arr = new byte[len];
int bitOffset = 0, index = 0;
for (int i = 0; i < source.Length; i++)
{
// note: this encodes little-endian
int val = source[i] & 2047;
int bitsLeft = 11;
if(bitOffset != 0)
{
val = val << bitOffset;
arr[index++] |= (byte)val;
bitsLeft -= (8 - bitOffset);
val >>= 8;
}
if(bitsLeft >= 8)
{
arr[index++] = (byte)val;
bitsLeft -= 8;
val >>= 8;
}
if(bitsLeft != 0)
{
arr[index] = (byte)val;
}
bitOffset = bitsLeft;
}
return arr;
}
private static int[] Decode(byte[] source)
{
int bits = source.Length * 8;
int len = (int)(bits / 11);
// note no need to worry about remaining chunks - no ambiguity since 11 > 8
int[] arr = new int[len];
int bitOffset = 0, index = 0;
for(int i = 0; i < source.Length; i++)
{
int val = source[i] << bitOffset;
int bitsLeftInVal = 11 - bitOffset;
if(bitsLeftInVal > 8)
{
arr[index] |= val;
bitOffset += 8;
}
else if(bitsLeftInVal == 8)
{
arr[index++] |= val;
bitOffset = 0;
}
else
{
arr[index++] |= (val & 2047);
if(index != arr.Length) arr[index] = val >> 11;
bitOffset = 8 - bitsLeftInVal;
}
}
return arr;
}
}
If you need a different layout you'll need to tweak it.
This encodes 512 MiB in just over a second on my machine.
Overview to the Encode method:
The first thing is does is pre-calculate the amount of space that is going to be required, and allocate the output buffer; since each input contributes 11 bits to the output, this is just some modulo math:
long bits = source.Length * 11;
int len = (int)(bits / 8);
if ((bits % 8) != 0) len++;
byte[] arr = new byte[len];
We know the output position won't match the input, and we know we're going to be starting each 11-bit chunk at different positions in bytes each time, so allocate variables for those, and loop over the input:
int bitOffset = 0, index = 0;
for (int i = 0; i < source.Length; i++)
{
...
}
return arr;
So: taking each input in turn (where the input is the value at position i), take the low 11 bits of the value - and observe that we have 11 bits (of this value) still to write:
int val = source[i] & 2047;
int bitsLeft = 11;
Now, if the current output value is partially written (i.e. bitOffset != 0), we should deal with that first. The amount of space left in the current output is 8 - bitOffset. Since we always have 11 input bits we don't need to worry about having more space than values to fill, so: left-shift our value by bitOffset (pads on the right with bitOffset zeros, as a binary operation), and "or" the lowest 8 bits of this with the output byte. Essentially this says "if bitOffset is 3, write the 5 low bits of val into the 5 high bits of the output buffer"; finally, fixup the values: increment our write position, record that we have fewer bits of the current value still to write, and use right-shift to discard the 8 low bits of val (which is made of bitOffset zeros and 8 - bitOffset "real" bits):
if(bitOffset != 0)
{
val = val << bitOffset;
arr[index++] |= (byte)val;
bitsLeft -= (8 - bitOffset);
val >>= 8;
}
The next question is: do we have (at least) an entire byte of data left? We might not, if bitOffset was 1 for example (so we'll have written 7 bits already, leaving just 4). If we do, we can just stamp that down and increment the write position - then once again track how many are left and throw away the low 8 bits:
if(bitsLeft >= 8)
{
arr[index++] = (byte)val;
bitsLeft -= 8;
val >>= 8;
}
And it is possible that we've still got some left-over; for example, if bitOffset was 7 we'll have written 1 bit in the first chunk, 8 bits in the second, leaving 2 more to write - or if bitOffset was 0 we won't have written anything in the first chunk, 8 in the second, leaving 3 left to write. So, stamp down whatever is left, but do not increment the write position - we've written to the low bits, but the next value might need to write to the high bits. Finally, update bitOffset to be however many low bits we wrote in the last step (which could be zero):
if(bitsLeft != 0)
{
arr[index] = (byte)val;
}
bitOffset = bitsLeft;
The Decode operation is the reverse of this logic - again, calculate the sizes and prepare the state:
int bits = source.Length * 8;
int len = (int)(bits / 11);
int[] arr = new int[len];
int bitOffset = 0, index = 0;
Now loop over the input:
for(int i = 0; i < source.Length; i++)
{
...
}
return arr;
Now, bitOffset is the start position that we want to write to in the current 11-bit value, so if we start at the start, it will be 0 on the first byte, then 8; 3 bits of the second byte join with the first 11-bit integer, so the 5 bits become part of the second - so bitOffset is 5 on the 3rd byte, etc. We can calculate the number of bits left in the current integer by subtracting from 11:
int val = source[i] << bitOffset;
int bitsLeftInVal = 11 - bitOffset;
Now we have 3 possible scenarios:
1) if we have more than 8 bits left in the current value, we can stamp down our input (as a bitwise "or") but do not increment the write position (as we have more to write for this value), and note that we're 8-bits further along:
if(bitsLeftInVal > 8)
{
arr[index] |= val;
bitOffset += 8;
}
2) if we have exactly 8 bits left in the current value, we can stamp down our input (as a bitwise "or") and increment the write position; the next loop can start at zero:
else if(bitsLeftInVal == 8)
{
arr[index++] |= val;
bitOffset = 0;
}
3) otherwise, we have less than 8 bits left in the current value; so we need to write the first bitsLeftInVal bits to the current output position (incrementing the output position), and whatever is left to the next output position. Since we already left-shifted by bitOffset, what this really means is simply: stamp down (as a bitwise "or") the low 11 bits (val & 2047) to the current position, and whatever is left (val >> 11) to the next if that wouldn't exceed our output buffer (padding zeros). Then calculate our new bitOffset:
else
{
arr[index++] |= (val & 2047);
if(index != arr.Length) arr[index] = val >> 11;
bitOffset = 8 - bitsLeftInVal;
}
And that's basically it. Lots of bitwise operations - shifts (<< / >>), masks (&) and combinations (|).
If you wanted to store the least significant 11 bits of an int into two bytes such that the least significant byte has bits 1-8 inclusive and the most significant byte has 9-11:
int toStore = 123456789;
byte msb = (byte) ((toStore >> 8) & 7); //or 0b111
byte lsb = (byte) (toStore & 255); //or 0b11111111
To check this, 123456789 in binary is:
0b111010110111100110100010101
MMMLLLLLLLL
The bits above L are lsb, and have a value of 21, above M are msb and have a value of 5
Doing the work is the shift operator >> where all the binary digits are slid to the right 8 places (8 of them disappear from the right hand side - they're gone, into oblivion):
0b111010110111100110100010101 >> 8 =
0b1110101101111001101
And the mask operator & (the mask operator works by only keeping bits where, in each position, they're 1 in the value and also 1 in the mask) :
0b111010110111100110100010101 &
0b000000000000000000011111111 (255) =
0b000000000000000000000010101
If you're processing an int array, just do this in a loop:
byte[] bs = new byte[ intarray.Length*2 ];
for(int x = 0, b=0; x < intarray.Length; x++){
int toStore = intarray[x];
bs[b++] = (byte) ((toStore >> 8) & 7);
bs[b++] = (byte) (toStore & 255);
}
Let say we have two bitmaps that are represented by unsigned long(64-bit) arrays. And I want to merge this two bitmaps using specific shift(offset).
For example merge bitmap1(bigger) into bitmap2(smaller) starting offset 3. Offset 3 mean that 3rd bit of bitmap1 corresponds to 0 bit of bitmap2.
By merge I mean logical Or operation. What is the cleanest way to do this?
Currently I have done this with simple uneffective for loop
const ulong BitsPerUlong = 64;
MergeAt(ulong startIndex, Bitmap bitmap2)
{
for (int i = startIndex; i < bitmap2.Capacity; i++)
{
bool newVal = bitmap2.GetAt(i) | bitmap1.GetAt(i)
bitmap2.SetAt(i, newVal)
}
}
bool GetAt(ulong index)
{
var dataOffset = BitOffsetToUlongOffset(index);
ulong mask = 0x1ul << ((int)(index % BitsPerUlong));
return (_data[dataOffset] & mask) == mask;
}
void SetAt(ulong index, bool value)
{
var dataOffset = BitOffsetToUlongOffset(index);
ulong mask = 0x1ul << ((int)(index % BitsPerUlong));
if (value)
{
_data[dataOffset] |= mask;
}
else
{
_data[dataOffset] &= ~mask;
}
}
ulong BitOffsetToUlongOffset(ulong index)
{
var dataOffset = index / BitsPerUlong;
return dataOffset;
}
(C/C++/C# accepted).
As you probably figured out yourself, if offset < BitsPerULong the first block can be merged with:
data1[0] |= data2[0] << offset;
Which leaves some bits in data2[0] unmerged, but you can get those with:
data2[0] >> (BitsPerULong - offset)
So the next merge for i > 0 becomes:
data1[i] |= (data2[i] << offset) | (data2[i-1] >> (BitsPerULong - offset));
from which you can construct a for-loop to merge all data. Of course, this still means a couple of bits from data2 will "fall off" but I think that's inherent to your problem description?
If you need a more generic solution where offset can also be greater than BitsPerULong, this needs a bit more work.
I presume you mean that you want to "merge" the smaller INTO the bigger.
Have you tried: bitmapLarger |= ( bitmapSmaller << 3 ) ?
I have problem with inserting a bit in number. I have a position where to insert this 1 bit. For example if i have a number 186 (represent in bits: 1011 1010) and I have a position 4. It will looks 1 0111 1010. And my second problem is almost the same as first but the operation is remove 1 bit in given position. If i have position 5, the number in bits will look 101 1010. Thank you
Keep calm and learn how to
I needed to do exactly the same for my Z80.NET project and that's how I solved it:
public static class NumberUtils
{
public static byte WithBit(this byte number, int bitPosition, bool value)
{
if(bitPosition < 0 || bitPosition > 7)
throw new InvalidOperationException("bit position must be between 0 and 7");
if(value)
return (byte)(number | (1 << bitPosition));
else
return (byte)(number & ~(1 << bitPosition));
}
}
Unit test:
[Test]
public void SetBit_works_for_middle_bit()
{
byte bit4reset = 0xFF;
Assert.AreEqual(0xEF, bit4reset.WithBit(4, false));
byte bit4set = 0x00;
Assert.AreEqual(0x10, bit4set.WithBit(4, true));
}
You can use System.Collections.BitArray for this task, it allows you to convert a number to a bit array, then manipulate individual bits of this number, then convert it back. Here's how you could insert a bit into a number at a certain position:
public static byte insertIntoPosition(byte number, int position, bool bit)
{
// converting the number to BitArray
BitArray a = new BitArray (new byte[]{number});
// shifting the bits to the left of the position
for (int j = a.Count - 1; j > position; j--) {
a.Set (j, a.Get (j - 1));
}
// setting the position bit
a.Set (position, bit);
// converting BitArray to byte again
byte[] array = new byte[1];
a.CopyTo(array, 0);
return array[0];
}
Removing a bit from a certain position is also easily done this way.
The following code will insert a single bit without the inefficiency of converting to a bit array with a for loop (turning a O(1) operation into a O(N) one).
public static int InsertBit(int input, int pos, bool state) {
//Split the input into two parts, one shifted and one not
int bottom = input;
int top = (input << 1);
//insert a '0' or '1' before the shifted part
if (state) top |= (1 << pos);
else top &= (~(1 << pos));
//keep the top bits of top
top &= (-1 << pos);
//keep the bottom bits of bottom
bottom &= ~(-1 << pos);
//combine the two parts.
return (bottom | top);
end;
Note that is is a translation of https://www.topbug.net/blog/2014/11/16/how-to-insert-1-bit-into-an-integer/
So not my own code.
Removing a bit is even easier:
public static int RemoveBit(int input, int pos) {
//Split the input into two parts, one shifted and one not
int bottom = input;
int top = (input >> 1);
//keep the top bits of x
top &= (-1 << pos);
//keep the bottom bits of y
bottom &= ~(-1 << pos);
//combine the two parts.
return (bottom | top);
end;
Note that remove bit will preserve the sign of input. If input is negative then it will insert 1's into the most significant bit. If you don't want that you'll have to make input an unsigned int.
Hi I am trying to use the custom Binary Integer division method:
Source: http://www.informit.com/guides/content.aspx?g=dotnet&seqNum=642
public static void DivMod (Int128 dividend, Int128 divisor, out Int128 quotient, out Int128 remainder)
{
// Determine the sign of the results and make the operands positive.
int remainderSign = 1;
int quotientSign = 1;
if (dividend < 0)
{
dividend = -dividend;
remainderSign = -1;
}
if (divisor < 0)
{
divisor = -divisor;
quotientSign = -1;
}
quotientSign *= remainderSign;
quotient = dividend;
remainder = 0;
for (int i = 0; i < 128; i++)
{
// Left shift Remainder:Quotient by 1
remainder <<= 1;
if (quotient < 0)
remainder._lo |= 1;
quotient <<= 1;
if (remainder >= divisor)
{
remainder -= divisor;
quotient++;
}
}
// Adjust sign of the results.
quotient *= quotientSign;
remainder *= remainderSign;
}
However I have 2 problems:
1) I would like to use it for 32 bit integers not Int128. so I assume that the Int128 should be replaced by int, and the (int i = 0; i < 128; i++) should be replaced by i < 32;. Correct?
2) remainder._lo |= 1 -> this line does not work at all in C#. I suppose it has to do with that custom 128bit int struct they use, and I have no idea what it is meant to do. Can somebody help me out with this one, and translate it so that it works with int32?
EDIT: just to clarify I know what the bitwise operators do, the problem part is this:
remainder._lo. I dont know what this property refers to, and not sure of the purpose of this line, and how it would be translated to an int32?
To use it with 32 bit integer (System.Int32) you can replace Int128 with int and the 128 in the for loop with 32 - So this is correct.
The _lo property is just the lower 64 bits of the 128 bits number. It is used because the biggest integer type in .NET is 64 bits (System.Int64) - So with 32 bits you can just omit the property:
remainder |= 1;
If you follow the link you gave in your question and go back a few pages you will find the actual implementation of the Int128 struct. It starts here.
It is explained on this page of the guide:
public struct Int128 : IComparable, IFormattable, IConvertible, IComparable<Int128_done>, IEquatable<Int128_done>
{
private ulong _lo;
private long _hi;
public Int128(UInt64 low, Int64 high)
{
_lo = low;
_hi = high;
}
}
You can ignore it with 32 bit integers, and just do some32int |= 1.
He says to loop once for each bit, so with a 32 bit integer you would loop only 32 times.
I want to convert an int to a byte[2] array using BCD.
The int in question will come from DateTime representing the Year and must be converted to two bytes.
Is there any pre-made function that does this or can you give me a simple way of doing this?
example:
int year = 2010
would output:
byte[2]{0x20, 0x10};
static byte[] Year2Bcd(int year) {
if (year < 0 || year > 9999) throw new ArgumentException();
int bcd = 0;
for (int digit = 0; digit < 4; ++digit) {
int nibble = year % 10;
bcd |= nibble << (digit * 4);
year /= 10;
}
return new byte[] { (byte)((bcd >> 8) & 0xff), (byte)(bcd & 0xff) };
}
Beware that you asked for a big-endian result, that's a bit unusual.
Use this method.
public static byte[] ToBcd(int value){
if(value<0 || value>99999999)
throw new ArgumentOutOfRangeException("value");
byte[] ret=new byte[4];
for(int i=0;i<4;i++){
ret[i]=(byte)(value%10);
value/=10;
ret[i]|=(byte)((value%10)<<4);
value/=10;
}
return ret;
}
This is essentially how it works.
If the value is less than 0 or greater than 99999999, the value won't fit in four bytes. More formally, if the value is less than 0 or is 10^(n*2) or greater, where n is the number of bytes, the value won't fit in n bytes.
For each byte:
Set that byte to the remainder of the value-divided-by-10 to the byte. (This will place the last digit in the low nibble [half-byte] of the current byte.)
Divide the value by 10.
Add 16 times the remainder of the value-divided-by-10 to the byte. (This will place the now-last digit in the high nibble of the current byte.)
Divide the value by 10.
(One optimization is to set every byte to 0 beforehand -- which is implicitly done by .NET when it allocates a new array -- and to stop iterating when the value reaches 0. This latter optimization is not done in the code above, for simplicity. Also, if available, some compilers or assemblers offer a divide/remainder routine that allows retrieving the quotient and remainder in one division step, an optimization which is not usually necessary though.)
Here's a terrible brute-force version. I'm sure there's a better way than this, but it ought to work anyway.
int digitOne = year / 1000;
int digitTwo = (year - digitOne * 1000) / 100;
int digitThree = (year - digitOne * 1000 - digitTwo * 100) / 10;
int digitFour = year - digitOne * 1000 - digitTwo * 100 - digitThree * 10;
byte[] bcdYear = new byte[] { digitOne << 4 | digitTwo, digitThree << 4 | digitFour };
The sad part about it is that fast binary to BCD conversions are built into the x86 microprocessor architecture, if you could get at them!
Here is a slightly cleaner version then Jeffrey's
static byte[] IntToBCD(int input)
{
if (input > 9999 || input < 0)
throw new ArgumentOutOfRangeException("input");
int thousands = input / 1000;
int hundreds = (input -= thousands * 1000) / 100;
int tens = (input -= hundreds * 100) / 10;
int ones = (input -= tens * 10);
byte[] bcd = new byte[] {
(byte)(thousands << 4 | hundreds),
(byte)(tens << 4 | ones)
};
return bcd;
}
maybe a simple parse function containing this loop
i=0;
while (id>0)
{
twodigits=id%100; //need 2 digits per byte
arr[i]=twodigits%10 + twodigits/10*16; //first digit on first 4 bits second digit shifted with 4 bits
id/=100;
i++;
}
More common solution
private IEnumerable<Byte> GetBytes(Decimal value)
{
Byte currentByte = 0;
Boolean odd = true;
while (value > 0)
{
if (odd)
currentByte = 0;
Decimal rest = value % 10;
value = (value-rest)/10;
currentByte |= (Byte)(odd ? (Byte)rest : (Byte)((Byte)rest << 4));
if(!odd)
yield return currentByte;
odd = !odd;
}
if(!odd)
yield return currentByte;
}
Same version as Peter O. but in VB.NET
Public Shared Function ToBcd(ByVal pValue As Integer) As Byte()
If pValue < 0 OrElse pValue > 99999999 Then Throw New ArgumentOutOfRangeException("value")
Dim ret As Byte() = New Byte(3) {} 'All bytes are init with 0's
For i As Integer = 0 To 3
ret(i) = CByte(pValue Mod 10)
pValue = Math.Floor(pValue / 10.0)
ret(i) = ret(i) Or CByte((pValue Mod 10) << 4)
pValue = Math.Floor(pValue / 10.0)
If pValue = 0 Then Exit For
Next
Return ret
End Function
The trick here is to be aware that simply using pValue /= 10 will round the value so if for instance the argument is "16", the first part of the byte will be correct, but the result of the division will be 2 (as 1.6 will be rounded up). Therefore I use the Math.Floor method.
I made a generic routine posted at IntToByteArray that you could use like:
var yearInBytes = ConvertBigIntToBcd(2010, 2);
static byte[] IntToBCD(int input) {
byte[] bcd = new byte[] {
(byte)(input>> 8),
(byte)(input& 0x00FF)
};
return bcd;
}