We Keep Coding

fast way to convert integer array to byte array (11 bit)

I have integer array and I need to convert it to byte array
but I need to take (only and just only) first 11 bit of each element of the هinteger array
and then convert it to a byte array
I tried this code
// ***********convert integer values to byte values
//***********to avoid the left zero padding on the byte array
// *********** first step : convert to binary string
// ***********second step : convert binary string to byte array
// *********** first step
string ByteString = Convert.ToString(IntArray[0], 2).PadLeft(11,'0');
for (int i = 1; i < IntArray.Length; i++)
ByteString = ByteString + Convert.ToString(IntArray[i], 2).PadLeft(11, '0');
// ***********second step
int numOfBytes = ByteString.Length / 8;
byte[] bytes = new byte[numOfBytes];
for (int i = 0; i < numOfBytes; ++i)
{
bytes[i] = Convert.ToByte(ByteString.Substring(8 * i, 8), 2);
}
But it takes too long time (if the file size large , the code takes more than 1 minute)
I need a very very fast code (very few milliseconds only )
can any one help me ?

Basically, you're going to be doing a lot of shifting and masking. The exact nature of that depends on the layout you want. If we assume that we pack little-endian from each int, appending on the left, so two 11-bit integers with positions:
abcdefghijk lmnopqrstuv
become the 8-bit chunks:
defghijk rstuvabc 00lmnopq
(i.e. take the lowest 8 bits of the first integer, which leaves 3 left over, so pack those into the low 3 bits of the next byte, then take the lowest 5 bits of the second integer, then finally the remaining 6 bits, padding with zero), then something like this should work:
using System;
using System.Linq;
static class Program
{
static string AsBinary(int val) => Convert.ToString(val, 2).PadLeft(11, '0');
static string AsBinary(byte val) => Convert.ToString(val, 2).PadLeft(8, '0');
static void Main()
{
int[] source = new int[1432];
var rand = new Random(123456);
for (int i = 0; i < source.Length; i++)
source[i] = rand.Next(0, 2047); // 11 bits
// Console.WriteLine(string.Join(" ", source.Take(5).Select(AsBinary)));
var raw = Encode(source);
// Console.WriteLine(string.Join(" ", raw.Take(6).Select(AsBinary)));
var clone = Decode(raw);
// now prove that it worked OK
if (source.Length != clone.Length)
{
Console.WriteLine($"Length: {source.Length} vs {clone.Length}");
}
else
{
int failCount = 0;
for (int i = 0; i < source.Length; i++)
{
if (source[i] != clone[i] && failCount++ == 0)
{
Console.WriteLine($"{i}: {source[i]} vs {clone[i]}");
}
}
Console.WriteLine($"Errors: {failCount}");
}
}
static byte[] Encode(int[] source)
{
long bits = source.Length * 11;
int len = (int)(bits / 8);
if ((bits % 8) != 0) len++;
byte[] arr = new byte[len];
int bitOffset = 0, index = 0;
for (int i = 0; i < source.Length; i++)
{
// note: this encodes little-endian
int val = source[i] & 2047;
int bitsLeft = 11;
if(bitOffset != 0)
{
val = val << bitOffset;
arr[index++] |= (byte)val;
bitsLeft -= (8 - bitOffset);
val >>= 8;
}
if(bitsLeft >= 8)
{
arr[index++] = (byte)val;
bitsLeft -= 8;
val >>= 8;
}
if(bitsLeft != 0)
{
arr[index] = (byte)val;
}
bitOffset = bitsLeft;
}
return arr;
}
private static int[] Decode(byte[] source)
{
int bits = source.Length * 8;
int len = (int)(bits / 11);
// note no need to worry about remaining chunks - no ambiguity since 11 > 8
int[] arr = new int[len];
int bitOffset = 0, index = 0;
for(int i = 0; i < source.Length; i++)
{
int val = source[i] << bitOffset;
int bitsLeftInVal = 11 - bitOffset;
if(bitsLeftInVal > 8)
{
arr[index] |= val;
bitOffset += 8;
}
else if(bitsLeftInVal == 8)
{
arr[index++] |= val;
bitOffset = 0;
}
else
{
arr[index++] |= (val & 2047);
if(index != arr.Length) arr[index] = val >> 11;
bitOffset = 8 - bitsLeftInVal;
}
}
return arr;
}
}
If you need a different layout you'll need to tweak it.
This encodes 512 MiB in just over a second on my machine.
Overview to the Encode method:
The first thing is does is pre-calculate the amount of space that is going to be required, and allocate the output buffer; since each input contributes 11 bits to the output, this is just some modulo math:
long bits = source.Length * 11;
int len = (int)(bits / 8);
if ((bits % 8) != 0) len++;
byte[] arr = new byte[len];
We know the output position won't match the input, and we know we're going to be starting each 11-bit chunk at different positions in bytes each time, so allocate variables for those, and loop over the input:
int bitOffset = 0, index = 0;
for (int i = 0; i < source.Length; i++)
{
...
}
return arr;
So: taking each input in turn (where the input is the value at position i), take the low 11 bits of the value - and observe that we have 11 bits (of this value) still to write:
int val = source[i] & 2047;
int bitsLeft = 11;
Now, if the current output value is partially written (i.e. bitOffset != 0), we should deal with that first. The amount of space left in the current output is 8 - bitOffset. Since we always have 11 input bits we don't need to worry about having more space than values to fill, so: left-shift our value by bitOffset (pads on the right with bitOffset zeros, as a binary operation), and "or" the lowest 8 bits of this with the output byte. Essentially this says "if bitOffset is 3, write the 5 low bits of val into the 5 high bits of the output buffer"; finally, fixup the values: increment our write position, record that we have fewer bits of the current value still to write, and use right-shift to discard the 8 low bits of val (which is made of bitOffset zeros and 8 - bitOffset "real" bits):
if(bitOffset != 0)
{
val = val << bitOffset;
arr[index++] |= (byte)val;
bitsLeft -= (8 - bitOffset);
val >>= 8;
}
The next question is: do we have (at least) an entire byte of data left? We might not, if bitOffset was 1 for example (so we'll have written 7 bits already, leaving just 4). If we do, we can just stamp that down and increment the write position - then once again track how many are left and throw away the low 8 bits:
if(bitsLeft >= 8)
{
arr[index++] = (byte)val;
bitsLeft -= 8;
val >>= 8;
}
And it is possible that we've still got some left-over; for example, if bitOffset was 7 we'll have written 1 bit in the first chunk, 8 bits in the second, leaving 2 more to write - or if bitOffset was 0 we won't have written anything in the first chunk, 8 in the second, leaving 3 left to write. So, stamp down whatever is left, but do not increment the write position - we've written to the low bits, but the next value might need to write to the high bits. Finally, update bitOffset to be however many low bits we wrote in the last step (which could be zero):
if(bitsLeft != 0)
{
arr[index] = (byte)val;
}
bitOffset = bitsLeft;
The Decode operation is the reverse of this logic - again, calculate the sizes and prepare the state:
int bits = source.Length * 8;
int len = (int)(bits / 11);
int[] arr = new int[len];
int bitOffset = 0, index = 0;
Now loop over the input:
for(int i = 0; i < source.Length; i++)
{
...
}
return arr;
Now, bitOffset is the start position that we want to write to in the current 11-bit value, so if we start at the start, it will be 0 on the first byte, then 8; 3 bits of the second byte join with the first 11-bit integer, so the 5 bits become part of the second - so bitOffset is 5 on the 3rd byte, etc. We can calculate the number of bits left in the current integer by subtracting from 11:
int val = source[i] << bitOffset;
int bitsLeftInVal = 11 - bitOffset;
Now we have 3 possible scenarios:
1) if we have more than 8 bits left in the current value, we can stamp down our input (as a bitwise "or") but do not increment the write position (as we have more to write for this value), and note that we're 8-bits further along:
if(bitsLeftInVal > 8)
{
arr[index] |= val;
bitOffset += 8;
}
2) if we have exactly 8 bits left in the current value, we can stamp down our input (as a bitwise "or") and increment the write position; the next loop can start at zero:
else if(bitsLeftInVal == 8)
{
arr[index++] |= val;
bitOffset = 0;
}
3) otherwise, we have less than 8 bits left in the current value; so we need to write the first bitsLeftInVal bits to the current output position (incrementing the output position), and whatever is left to the next output position. Since we already left-shifted by bitOffset, what this really means is simply: stamp down (as a bitwise "or") the low 11 bits (val & 2047) to the current position, and whatever is left (val >> 11) to the next if that wouldn't exceed our output buffer (padding zeros). Then calculate our new bitOffset:
else
{
arr[index++] |= (val & 2047);
if(index != arr.Length) arr[index] = val >> 11;
bitOffset = 8 - bitsLeftInVal;
}
And that's basically it. Lots of bitwise operations - shifts (<< / >>), masks (&) and combinations (|).

If you wanted to store the least significant 11 bits of an int into two bytes such that the least significant byte has bits 1-8 inclusive and the most significant byte has 9-11:
int toStore = 123456789;
byte msb = (byte) ((toStore >> 8) & 7); //or 0b111
byte lsb = (byte) (toStore & 255); //or 0b11111111
To check this, 123456789 in binary is:
0b111010110111100110100010101
MMMLLLLLLLL
The bits above L are lsb, and have a value of 21, above M are msb and have a value of 5
Doing the work is the shift operator >> where all the binary digits are slid to the right 8 places (8 of them disappear from the right hand side - they're gone, into oblivion):
0b111010110111100110100010101 >> 8 =
0b1110101101111001101
And the mask operator & (the mask operator works by only keeping bits where, in each position, they're 1 in the value and also 1 in the mask) :
0b111010110111100110100010101 &
0b000000000000000000011111111 (255) =
0b000000000000000000000010101
If you're processing an int array, just do this in a loop:
byte[] bs = new byte[ intarray.Length*2 ];
for(int x = 0, b=0; x < intarray.Length; x++){
int toStore = intarray[x];
bs[b++] = (byte) ((toStore >> 8) & 7);
bs[b++] = (byte) (toStore & 255);
}

Insert 1 bit in number c#

I have problem with inserting a bit in number. I have a position where to insert this 1 bit. For example if i have a number 186 (represent in bits: 1011 1010) and I have a position 4. It will looks 1 0111 1010. And my second problem is almost the same as first but the operation is remove 1 bit in given position. If i have position 5, the number in bits will look 101 1010. Thank you
Keep calm and learn how to

I needed to do exactly the same for my Z80.NET project and that's how I solved it:
public static class NumberUtils
{
public static byte WithBit(this byte number, int bitPosition, bool value)
{
if(bitPosition < 0 || bitPosition > 7)
throw new InvalidOperationException("bit position must be between 0 and 7");
if(value)
return (byte)(number | (1 << bitPosition));
else
return (byte)(number & ~(1 << bitPosition));
}
}
Unit test:
[Test]
public void SetBit_works_for_middle_bit()
{
byte bit4reset = 0xFF;
Assert.AreEqual(0xEF, bit4reset.WithBit(4, false));
byte bit4set = 0x00;
Assert.AreEqual(0x10, bit4set.WithBit(4, true));
}

You can use System.Collections.BitArray for this task, it allows you to convert a number to a bit array, then manipulate individual bits of this number, then convert it back. Here's how you could insert a bit into a number at a certain position:
public static byte insertIntoPosition(byte number, int position, bool bit)
{
// converting the number to BitArray
BitArray a = new BitArray (new byte[]{number});
// shifting the bits to the left of the position
for (int j = a.Count - 1; j > position; j--) {
a.Set (j, a.Get (j - 1));
}
// setting the position bit
a.Set (position, bit);
// converting BitArray to byte again
byte[] array = new byte[1];
a.CopyTo(array, 0);
return array[0];
}
Removing a bit from a certain position is also easily done this way.

The following code will insert a single bit without the inefficiency of converting to a bit array with a for loop (turning a O(1) operation into a O(N) one).
public static int InsertBit(int input, int pos, bool state) {
//Split the input into two parts, one shifted and one not
int bottom = input;
int top = (input << 1);
//insert a '0' or '1' before the shifted part
if (state) top |= (1 << pos);
else top &= (~(1 << pos));
//keep the top bits of top
top &= (-1 << pos);
//keep the bottom bits of bottom
bottom &= ~(-1 << pos);
//combine the two parts.
return (bottom | top);
end;
Note that is is a translation of https://www.topbug.net/blog/2014/11/16/how-to-insert-1-bit-into-an-integer/
So not my own code.
Removing a bit is even easier:
public static int RemoveBit(int input, int pos) {
//Split the input into two parts, one shifted and one not
int bottom = input;
int top = (input >> 1);
//keep the top bits of x
top &= (-1 << pos);
//keep the bottom bits of y
bottom &= ~(-1 << pos);
//combine the two parts.
return (bottom | top);
end;
Note that remove bit will preserve the sign of input. If input is negative then it will insert 1's into the most significant bit. If you don't want that you'll have to make input an unsigned int.

Converting Ada code to C#

So I need help converting this ada code int c#, it's basically a checksum algorithm.
ADA:
CHECKSUM_VALUE := ((ROTATE_LEFT_1_BIT(CHECKSUM_VALUE)) xor (CURRENT_VALUE));
This is what I could come up with:
C#:
checksum = RotateLeft(checksum, rotateCount, sizeof(ushort) * 8) ^ word;
RotateLeft Function:
public static int RotateLeft(int value, ushort rotateCount, int dataSize)
{
return (value << rotateCount) | (value >> (dataSize - rotateCount));
}
However when comparing the checksum results from the ada and C# algorithms, they do not match so I think my conversion isn't correct, anyone who has used ada before can give some input would be really helpful.
Thanks

The issue seems to be with the C# and perhaps not with your interpretation of the ADA code. If you are truly rotating a 16 bit unsigned number as your post is implying, then you will need to mask the upper 2 bytes of the resulting integer value so that they do not contribute to the answer. Casting an uint x to ushort in C# will do the equivalent of x & 0x0000FFFF
public static ushort RotateLeft(ushort value, int count)
{
int left = value << count;
int right = value >> (16 - count);
return (ushort)(left | right);
}

This answer is in C, since I don’t have a C# compiler.
You have value as an int, which is signed, so that a right shift extends the sign bit into the vacated space; so in (value << rotateCount) | (value >> (dataSize - rotateCount)), the right-hand half ((value >> (dataSize - rotateCount))) needs to have the top bits masked off. And I don’t know why you need dataSize, isn’t it sizeof(value)?
I think a better solution would be to use unsigned, so that a right shift introduces zeros into the vacated space.
#include <stdio.h>
unsigned rotateLeft(unsigned value, int by) {
const unsigned bits = sizeof(value) * 8;
return (value << by) | (value >> (bits - by));
}
int main() {
unsigned input = 0x52525252;
unsigned result = input;
printf("input: %x\n", input);
{
int j;
for (j = 0; j < 8; j++) {
result = rotateLeft(result, 1);
printf("result: %x\n", result);
}
}
return 0;
}
The output is
input: 52525252
result: a4a4a4a4
result: 49494949
result: 92929292
result: 25252525
result: 4a4a4a4a
result: 94949494
result: 29292929
result: 52525252

Set a range of bits in a circular bit field

I have a bit field consisting of 64 bits:
long bitfield = 0;
I can set the bit for a given index as follows:
void Set(long index)
{
bitfield |= 1L << (int)(index % 64);
}
i.e. if the index is 0, 64, 128, ... then bit 0 is set, if the index is 1, 65, 129, ... then bit 1 is set, and so on.
Question: given an index and a count (or a lower and upper index), how can I set the bits for all indexes in this range without using a loop?

long SetRangeMask(int lower, int upper) // 3..7
{
if (! (lower <= upper)) throw new ArgumentException("...");
int size = upper - lower + 1; // 7 - 3 + 1 = 5
if (size >= 64) return -1;
long mask = (1 << size) - 1; // #00100000 - 1 = #000011111
return mask << lower | mask >> -lower; // #00011111 << 3 = #011111000
}

You could use a lookup table for combined bit masks
A real simple approach with no thought to special cases or optimizations like these questions raised, would look like:
static readonly private long[] maskLUT = new long[64,64] { /* generated */ };
void SetRange(long lobit, long hibit)
{
lobit %= 64;
hibit %= 64;
bitfield |= lobit<hibit? maskLUT[lobit,hibit] : maskLUT[hibit,lobit];
}
Thoughts:
you might consider an optimization that given [lobit...hibit], if hibit-lobit>=64 you can set all bits at once.
There is a bit of thought to be put in the connected-ness of regions given the fact that both boundaries can wrap around (do you wrap-around both boundaries first, or do you wraparound lobit, and use the delta to find the hibit from the wrapped boundary, like with the optimization mentioned before?)

You can use 2x-1 to create a mask x bits long, then shift it and OR it in, like so:
void Set( int index, int count ) {
bitfield |= (long)(Math.Pow( 2, count ) - 1) << ((index-count) % 64);
}
Update: I like to think that Math.Pow optimizes powers of two to a left shift, but it may not. If that's the case, you can get a little more performance by replacing the call to Math.Pow with the corresponding left shift:
public void Set( int index, int count ) {
bitfield |= ((2 << count - 1) - 1) << ((index-count) % 64);
}

C# Language: Changing the First Four Bits in a Byte

In order to utilize a byte to its fullest potential, I'm attempting to store two unique values into a byte: one in the first four bits and another in the second four bits. However, I've found that, while this practice allows for optimized memory allocation, it makes changing the individual values stored in the byte difficult.
In my code, I want to change the first set of four bits in a byte while maintaining the value of the second four bits in the same byte. While bitwise operations allow me to easily retrieve and manipulate the first four bit values, I'm finding it difficult to concatenate this new value with the second set of four bits in a byte. The question is, how can I erase the first four bits from a byte (or, more accurately, set them all the zero) and add the new set of 4 bits to replace the four bits that were just erased, thus preserving the last 4 bits in a byte while changing the first four?
Here's an example:
// Changes the first four bits in a byte to the parameter value
public void changeFirstFourBits(byte newFirstFour)
{
// If 'newFirstFour' is 0101 in binary, make 'value' 01011111 in binary, changing
// the first four bits but leaving the second four alone.
}
private byte value = 255; // binary: 11111111

Use bitwise AND (&) to clear out the old bits, shift the new bits to the correct position and bitwise OR (|) them together:
value = (value & 0xF) | (newFirstFour << 4);
Here's what happens:
value : abcdefgh
newFirstFour : 0000xyzw
0xF : 00001111
value & 0xF : 0000efgh
newFirstFour << 4 : xyzw0000
(value & 0xF) | (newFirstFour << 4) : xyzwefgh

When I have to do bit-twiddling like this, I make a readonly struct to do it for me. A four-bit integer is called nybble, of course:
struct TwoNybbles
{
private readonly byte b;
public byte High { get { return (byte)(b >> 4); } }
public byte Low { get { return (byte)(b & 0x0F); } {
public TwoNybbles(byte high, byte low)
{
this.b = (byte)((high << 4) | (low & 0x0F));
}
And then add implicit conversions between TwoNybbles and byte. Now you can just treat any byte as having a High and Low byte without putting all that ugly bit twiddling in your mainline code.

You first mask out you the high four bytes using value & 0xF. Then you shift the new bits to the high four bits using newFirstFour << 4 and finally you combine them together using binary or.
public void changeHighFourBits(byte newHighFour)
{
value=(byte)( (value & 0x0F) | (newFirstFour << 4));
}
public void changeLowFourBits(byte newLowFour)
{
value=(byte)( (value & 0xF0) | newLowFour);
}

I'm not really sure what your method there is supposed to do, but here are some methods for you:
void setHigh(ref byte b, byte val) {
b = (b & 0xf) | (val << 4);
}
byte high(byte b) {
return (b & 0xf0) >> 4;
}
void setLow(ref byte b, byte val) {
b = (b & 0xf0) | val;
}
byte low(byte b) {
return b & 0xf;
}
Should be self-explanatory.

public int SplatBit(int Reg, int Val, int ValLen, int Pos)
{
int mask = ((1 << ValLen) - 1) << Pos;
int newv = Val << Pos;
int res = (Reg & ~mask) | newv;
return res;
}
Example:
Reg = 135
Val = 9 (ValLen = 4, because 9 = 1001)
Pos = 2
135 = 10000111
9 = 1001
9 << Pos = 100100
Result = 10100111

A quick look would indicate that a bitwise and can be achieved using the & operator. So to remove the first four bytes you should be able to do:
byte value1=255; //11111111
byte value2=15; //00001111
return value1&value2;

Assuming newVal contains the value you want to store in origVal.
Do this for the 4 least significant bits:
byte origVal = ???;
byte newVal = ???
orig = (origVal & 0xF0) + newVal;
and this for the 4 most significant bits:
byte origVal = ???;
byte newVal = ???
orig = (origVal & 0xF) + (newVal << 4);

I know you asked specifically about clearing out the first four bits, which has been answered several times, but I wanted to point out that if you have two values <= decimal 15, you can combine them into 8 bits simply with this:
public int setBits(int upperFour, int lowerFour)
{
return upperFour << 4 | lowerFour;
}
The result will be xxxxyyyy where
xxxx = upperFour
yyyy = lowerFour
And that is what you seem to be trying to do.

Here's some code, but I think the earlier answers will do it for you. This is just to show some sort of test code to copy and past into a simple console project (the WriteBits method by be of help):
static void Main(string[] args)
{
int b1 = 255;
WriteBits(b1);
int b2 = b1 >> 4;
WriteBits(b2);
int b3 = b1 & ~0xF ;
WriteBits(b3);
// Store 5 in first nibble
int b4 = 5 << 4;
WriteBits(b4);
// Store 8 in second nibble
int b5 = 8;
WriteBits(b5);
// Store 5 and 8 in first and second nibbles
int b6 = 0;
b6 |= (5 << 4) + 8;
WriteBits(b6);
// Store 2 and 4
int b7 = 0;
b7 = StoreFirstNibble(2, b7);
b7 = StoreSecondNibble(4, b7);
WriteBits(b7);
// Read First Nibble
int first = ReadFirstNibble(b7);
WriteBits(first);
// Read Second Nibble
int second = ReadSecondNibble(b7);
WriteBits(second);
}
static int ReadFirstNibble(int storage)
{
return storage >> 4;
}
static int ReadSecondNibble(int storage)
{
return storage &= 0xF;
}
static int StoreFirstNibble(int val, int storage)
{
return storage |= (val << 4);
}
static int StoreSecondNibble(int val, int storage)
{
return storage |= val;
}
static void WriteBits(int b)
{
Console.WriteLine(BitConverter.ToString(BitConverter.GetBytes(b),0));
}
}