We are doing some performance optimizations in our project and with the profiler I came upon the following method:
private int CalculateAdcValues(byte lowIndex)
{
byte middleIndex = (byte)(lowIndex + 1);
byte highIndex = (byte)(lowIndex + 2);
// samples is a byte[]
retrun (int)((int)(samples[highIndex] << 24)
+ (int)(samples[middleIndex] << 16) + (int)(samples[lowIndex] << 8));
}
This method is already pretty fast with ~1µs per execution, but it is called ~100.000 times per second and so it takes ~10% of the CPU.
Does anyone have an idea how to further improve this method?
EDIT:
Current solution:
fixed (byte* p = samples)
{
for (; loopIndex < 61; loopIndex += 3)
{
adcValues[k++] = *((int*)(p + loopIndex)) << 8;
}
}
This takes <40% of the time then before (the "whole method" took ~35µs per call before and ~13µs now). The for-loop actualy takes more time then the calcualtion now...
I strongly suspect that after casting to byte, your indexes are being converted back to int anyway for use in the array indexing operation. That will be cheap, but may not be entirely free. So get rid of the casts, unless you were using the conversion to byte to effectively get the index within the range 0..255. At that point you can get rid of the separate local variables, too.
Additionally, your casts to int are no-ops as the shift operations are only defined on int and higher types.
Finally, using | may be faster than +:
private int CalculateAdcValues(byte lowIndex)
{
return (samples[lowIndex + 2] << 24) |
(samples[lowIndex + 1] << 16) |
(samples[lowIndex] << 8);
}
(Why is there nothing in the bottom 8 bits? Is that deliberate? Note that the result will end up being negative if samples[lowIndex + 2] has its top bit set - is that okay?)
Seeing that you have a friendly endianess, go unsafe
unsafe int CalculateAdcValuesFast1(int lowIndex)
{
fixed (byte* p = &samples[lowIndex])
{
return *(int*)p << 8;
}
}
On x86 about 30% faster. Not much gain as I hoped. About 40% when on x64.
As suggested by #CodeInChaos:
var bounds = samples.Length - 3;
fixed (byte* p = samples)
{
for (int i = 0; i < 1000000000; i++)
{
var r = CalculateAdcValuesFast2(p, i % bounds); // about 2x faster
// or inlined:
var r = *((int*)(p + i % bounds)) << 8; // about 3x faster
// do something
}
}
unsafe object CalculateAdcValuesFast2(byte* p1, int p2)
{
return *((int*)(p1 + p2)) << 8;
}
May be following can be little faster. I have removed casting to integer.
var middleIndex = (byte)(lowIndex + 1);
var highIndex = (byte)(lowIndex + 2);
return (this.samples[highIndex] << 24) + (this.samples[middleIndex] << 16) + (this.samples[lowIndex] << 8);
Related
I have been tasked to rewrite a small program written in C++ to C#.
But I came across this line that I couldn't understand fully. Is it concatenating the string length to the string or the pointer?
int n = _keyData * int(*(int*)(_chap + strlen(_chap) - 4));
This is the variables:
short _ver = 12;
short _keyData = short(_ver * _ver);
char _chap[100]; // Hold time with format [%02d-%02d %02d:%02d:%02d:%03d]
*(int*)(_chap + strlen(_chap) - 4) is a strict aliasing violation. Reinterpreting raw bytes as an int is type punning and is not allowed in C++ (even though some compilers tolerate it).
To fix it (assuming a little-endian system), you can rewrite it like this:
short _ver = 12;
short _keyData = short(_ver * _ver);
char _chap[100]; // Hold time with format [%02d-%02d %02d:%02d:%02d:%03d]
int len = strlen(_chap);
int x = (int)(((unsigned)_chap[len - 1] << 24) |
((unsigned)_chap[len - 2] << 16) |
((unsigned)_chap[len - 3] << 8) |
(unsigned)_chap[len - 4]);
int n = _keyData * x;
Coincidently, it should be easy to port this to C# now.
I have integer array and I need to convert it to byte array
but I need to take (only and just only) first 11 bit of each element of the هinteger array
and then convert it to a byte array
I tried this code
// ***********convert integer values to byte values
//***********to avoid the left zero padding on the byte array
// *********** first step : convert to binary string
// ***********second step : convert binary string to byte array
// *********** first step
string ByteString = Convert.ToString(IntArray[0], 2).PadLeft(11,'0');
for (int i = 1; i < IntArray.Length; i++)
ByteString = ByteString + Convert.ToString(IntArray[i], 2).PadLeft(11, '0');
// ***********second step
int numOfBytes = ByteString.Length / 8;
byte[] bytes = new byte[numOfBytes];
for (int i = 0; i < numOfBytes; ++i)
{
bytes[i] = Convert.ToByte(ByteString.Substring(8 * i, 8), 2);
}
But it takes too long time (if the file size large , the code takes more than 1 minute)
I need a very very fast code (very few milliseconds only )
can any one help me ?
Basically, you're going to be doing a lot of shifting and masking. The exact nature of that depends on the layout you want. If we assume that we pack little-endian from each int, appending on the left, so two 11-bit integers with positions:
abcdefghijk lmnopqrstuv
become the 8-bit chunks:
defghijk rstuvabc 00lmnopq
(i.e. take the lowest 8 bits of the first integer, which leaves 3 left over, so pack those into the low 3 bits of the next byte, then take the lowest 5 bits of the second integer, then finally the remaining 6 bits, padding with zero), then something like this should work:
using System;
using System.Linq;
static class Program
{
static string AsBinary(int val) => Convert.ToString(val, 2).PadLeft(11, '0');
static string AsBinary(byte val) => Convert.ToString(val, 2).PadLeft(8, '0');
static void Main()
{
int[] source = new int[1432];
var rand = new Random(123456);
for (int i = 0; i < source.Length; i++)
source[i] = rand.Next(0, 2047); // 11 bits
// Console.WriteLine(string.Join(" ", source.Take(5).Select(AsBinary)));
var raw = Encode(source);
// Console.WriteLine(string.Join(" ", raw.Take(6).Select(AsBinary)));
var clone = Decode(raw);
// now prove that it worked OK
if (source.Length != clone.Length)
{
Console.WriteLine($"Length: {source.Length} vs {clone.Length}");
}
else
{
int failCount = 0;
for (int i = 0; i < source.Length; i++)
{
if (source[i] != clone[i] && failCount++ == 0)
{
Console.WriteLine($"{i}: {source[i]} vs {clone[i]}");
}
}
Console.WriteLine($"Errors: {failCount}");
}
}
static byte[] Encode(int[] source)
{
long bits = source.Length * 11;
int len = (int)(bits / 8);
if ((bits % 8) != 0) len++;
byte[] arr = new byte[len];
int bitOffset = 0, index = 0;
for (int i = 0; i < source.Length; i++)
{
// note: this encodes little-endian
int val = source[i] & 2047;
int bitsLeft = 11;
if(bitOffset != 0)
{
val = val << bitOffset;
arr[index++] |= (byte)val;
bitsLeft -= (8 - bitOffset);
val >>= 8;
}
if(bitsLeft >= 8)
{
arr[index++] = (byte)val;
bitsLeft -= 8;
val >>= 8;
}
if(bitsLeft != 0)
{
arr[index] = (byte)val;
}
bitOffset = bitsLeft;
}
return arr;
}
private static int[] Decode(byte[] source)
{
int bits = source.Length * 8;
int len = (int)(bits / 11);
// note no need to worry about remaining chunks - no ambiguity since 11 > 8
int[] arr = new int[len];
int bitOffset = 0, index = 0;
for(int i = 0; i < source.Length; i++)
{
int val = source[i] << bitOffset;
int bitsLeftInVal = 11 - bitOffset;
if(bitsLeftInVal > 8)
{
arr[index] |= val;
bitOffset += 8;
}
else if(bitsLeftInVal == 8)
{
arr[index++] |= val;
bitOffset = 0;
}
else
{
arr[index++] |= (val & 2047);
if(index != arr.Length) arr[index] = val >> 11;
bitOffset = 8 - bitsLeftInVal;
}
}
return arr;
}
}
If you need a different layout you'll need to tweak it.
This encodes 512 MiB in just over a second on my machine.
Overview to the Encode method:
The first thing is does is pre-calculate the amount of space that is going to be required, and allocate the output buffer; since each input contributes 11 bits to the output, this is just some modulo math:
long bits = source.Length * 11;
int len = (int)(bits / 8);
if ((bits % 8) != 0) len++;
byte[] arr = new byte[len];
We know the output position won't match the input, and we know we're going to be starting each 11-bit chunk at different positions in bytes each time, so allocate variables for those, and loop over the input:
int bitOffset = 0, index = 0;
for (int i = 0; i < source.Length; i++)
{
...
}
return arr;
So: taking each input in turn (where the input is the value at position i), take the low 11 bits of the value - and observe that we have 11 bits (of this value) still to write:
int val = source[i] & 2047;
int bitsLeft = 11;
Now, if the current output value is partially written (i.e. bitOffset != 0), we should deal with that first. The amount of space left in the current output is 8 - bitOffset. Since we always have 11 input bits we don't need to worry about having more space than values to fill, so: left-shift our value by bitOffset (pads on the right with bitOffset zeros, as a binary operation), and "or" the lowest 8 bits of this with the output byte. Essentially this says "if bitOffset is 3, write the 5 low bits of val into the 5 high bits of the output buffer"; finally, fixup the values: increment our write position, record that we have fewer bits of the current value still to write, and use right-shift to discard the 8 low bits of val (which is made of bitOffset zeros and 8 - bitOffset "real" bits):
if(bitOffset != 0)
{
val = val << bitOffset;
arr[index++] |= (byte)val;
bitsLeft -= (8 - bitOffset);
val >>= 8;
}
The next question is: do we have (at least) an entire byte of data left? We might not, if bitOffset was 1 for example (so we'll have written 7 bits already, leaving just 4). If we do, we can just stamp that down and increment the write position - then once again track how many are left and throw away the low 8 bits:
if(bitsLeft >= 8)
{
arr[index++] = (byte)val;
bitsLeft -= 8;
val >>= 8;
}
And it is possible that we've still got some left-over; for example, if bitOffset was 7 we'll have written 1 bit in the first chunk, 8 bits in the second, leaving 2 more to write - or if bitOffset was 0 we won't have written anything in the first chunk, 8 in the second, leaving 3 left to write. So, stamp down whatever is left, but do not increment the write position - we've written to the low bits, but the next value might need to write to the high bits. Finally, update bitOffset to be however many low bits we wrote in the last step (which could be zero):
if(bitsLeft != 0)
{
arr[index] = (byte)val;
}
bitOffset = bitsLeft;
The Decode operation is the reverse of this logic - again, calculate the sizes and prepare the state:
int bits = source.Length * 8;
int len = (int)(bits / 11);
int[] arr = new int[len];
int bitOffset = 0, index = 0;
Now loop over the input:
for(int i = 0; i < source.Length; i++)
{
...
}
return arr;
Now, bitOffset is the start position that we want to write to in the current 11-bit value, so if we start at the start, it will be 0 on the first byte, then 8; 3 bits of the second byte join with the first 11-bit integer, so the 5 bits become part of the second - so bitOffset is 5 on the 3rd byte, etc. We can calculate the number of bits left in the current integer by subtracting from 11:
int val = source[i] << bitOffset;
int bitsLeftInVal = 11 - bitOffset;
Now we have 3 possible scenarios:
1) if we have more than 8 bits left in the current value, we can stamp down our input (as a bitwise "or") but do not increment the write position (as we have more to write for this value), and note that we're 8-bits further along:
if(bitsLeftInVal > 8)
{
arr[index] |= val;
bitOffset += 8;
}
2) if we have exactly 8 bits left in the current value, we can stamp down our input (as a bitwise "or") and increment the write position; the next loop can start at zero:
else if(bitsLeftInVal == 8)
{
arr[index++] |= val;
bitOffset = 0;
}
3) otherwise, we have less than 8 bits left in the current value; so we need to write the first bitsLeftInVal bits to the current output position (incrementing the output position), and whatever is left to the next output position. Since we already left-shifted by bitOffset, what this really means is simply: stamp down (as a bitwise "or") the low 11 bits (val & 2047) to the current position, and whatever is left (val >> 11) to the next if that wouldn't exceed our output buffer (padding zeros). Then calculate our new bitOffset:
else
{
arr[index++] |= (val & 2047);
if(index != arr.Length) arr[index] = val >> 11;
bitOffset = 8 - bitsLeftInVal;
}
And that's basically it. Lots of bitwise operations - shifts (<< / >>), masks (&) and combinations (|).
If you wanted to store the least significant 11 bits of an int into two bytes such that the least significant byte has bits 1-8 inclusive and the most significant byte has 9-11:
int toStore = 123456789;
byte msb = (byte) ((toStore >> 8) & 7); //or 0b111
byte lsb = (byte) (toStore & 255); //or 0b11111111
To check this, 123456789 in binary is:
0b111010110111100110100010101
MMMLLLLLLLL
The bits above L are lsb, and have a value of 21, above M are msb and have a value of 5
Doing the work is the shift operator >> where all the binary digits are slid to the right 8 places (8 of them disappear from the right hand side - they're gone, into oblivion):
0b111010110111100110100010101 >> 8 =
0b1110101101111001101
And the mask operator & (the mask operator works by only keeping bits where, in each position, they're 1 in the value and also 1 in the mask) :
0b111010110111100110100010101 &
0b000000000000000000011111111 (255) =
0b000000000000000000000010101
If you're processing an int array, just do this in a loop:
byte[] bs = new byte[ intarray.Length*2 ];
for(int x = 0, b=0; x < intarray.Length; x++){
int toStore = intarray[x];
bs[b++] = (byte) ((toStore >> 8) & 7);
bs[b++] = (byte) (toStore & 255);
}
Let say we have two bitmaps that are represented by unsigned long(64-bit) arrays. And I want to merge this two bitmaps using specific shift(offset).
For example merge bitmap1(bigger) into bitmap2(smaller) starting offset 3. Offset 3 mean that 3rd bit of bitmap1 corresponds to 0 bit of bitmap2.
By merge I mean logical Or operation. What is the cleanest way to do this?
Currently I have done this with simple uneffective for loop
const ulong BitsPerUlong = 64;
MergeAt(ulong startIndex, Bitmap bitmap2)
{
for (int i = startIndex; i < bitmap2.Capacity; i++)
{
bool newVal = bitmap2.GetAt(i) | bitmap1.GetAt(i)
bitmap2.SetAt(i, newVal)
}
}
bool GetAt(ulong index)
{
var dataOffset = BitOffsetToUlongOffset(index);
ulong mask = 0x1ul << ((int)(index % BitsPerUlong));
return (_data[dataOffset] & mask) == mask;
}
void SetAt(ulong index, bool value)
{
var dataOffset = BitOffsetToUlongOffset(index);
ulong mask = 0x1ul << ((int)(index % BitsPerUlong));
if (value)
{
_data[dataOffset] |= mask;
}
else
{
_data[dataOffset] &= ~mask;
}
}
ulong BitOffsetToUlongOffset(ulong index)
{
var dataOffset = index / BitsPerUlong;
return dataOffset;
}
(C/C++/C# accepted).
As you probably figured out yourself, if offset < BitsPerULong the first block can be merged with:
data1[0] |= data2[0] << offset;
Which leaves some bits in data2[0] unmerged, but you can get those with:
data2[0] >> (BitsPerULong - offset)
So the next merge for i > 0 becomes:
data1[i] |= (data2[i] << offset) | (data2[i-1] >> (BitsPerULong - offset));
from which you can construct a for-loop to merge all data. Of course, this still means a couple of bits from data2 will "fall off" but I think that's inherent to your problem description?
If you need a more generic solution where offset can also be greater than BitsPerULong, this needs a bit more work.
I presume you mean that you want to "merge" the smaller INTO the bigger.
Have you tried: bitmapLarger |= ( bitmapSmaller << 3 ) ?
I'm trying to compare two strings(Tx & Rx data) and find the quantity of unequal chars.
With the help of the following code, I managed to get the quantity,
string TxData = "00001111";
string RxData = "00000000";
int distorted = 0;
for (int i = 0; i < TxData.Length; i++)
{
if (TxData[i] != RxData[i])
distorted++;
}
Console.Write("Distorted Bits (qty) : {0}", distorted);
Result:
Distorted Bits (qty) : 4
But I'm very curious to know if there's any better way to do this task?
Thanks for your time...:)
If they're always the same length:
int distorted = TxData.Zip(RxData, (a,b) => a == b ? 0 : 1).Sum();
I like okrumnows answer by its simplisity, but assuming that you really already have bytes (or int) and don't need to convert them to string in the first place, you would probably be better of doing something like:
int myMethod(byte byte1, byte byte2)
{
//byte1 = Convert.ToByte("10010101",2);
//byte2 = Convert.ToByte("10011101",2);
byte xorvalue = (byte)( byte1 ^ byte2);
return NumberOfSetBits(xorvalue);
}
private static int NumberOfSetBits(uint i)
{
i = i - ((i >> 1) & 0x55555555);
i = (i & 0x33333333) + ((i >> 2) & 0x33333333);
return (((i + (i >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24;
}
This will be much faster.
I have two byte arrays with the same length. I need to perform XOR operation between each byte and after this calculate sum of bits.
For example:
11110000^01010101 = 10100101 -> so 1+1+1+1 = 4
I need do the same operation for each element in byte array.
Use a lookup table. There are only 256 possible values after XORing, so it's not exactly going to take a long time. Unlike izb's solution though, I wouldn't suggest manually putting all the values in though - compute the lookup table once at startup using one of the looping answers.
For example:
public static class ByteArrayHelpers
{
private static readonly int[] LookupTable =
Enumerable.Range(0, 256).Select(CountBits).ToArray();
private static int CountBits(int value)
{
int count = 0;
for (int i=0; i < 8; i++)
{
count += (value >> i) & 1;
}
return count;
}
public static int CountBitsAfterXor(byte[] array)
{
int xor = 0;
foreach (byte b in array)
{
xor ^= b;
}
return LookupTable[xor];
}
}
(You could make it an extension method if you really wanted...)
Note the use of byte[] in the CountBitsAfterXor method - you could make it an IEnumerable<byte> for more generality, but iterating over an array (which is known to be an array at compile-time) will be faster. Probably only microscopically faster, but hey, you asked for the fastest way :)
I would almost certainly actually express it as
public static int CountBitsAfterXor(IEnumerable<byte> data)
in real life, but see which works better for you.
Also note the type of the xor variable as an int. In fact, there's no XOR operator defined for byte values, and if you made xor a byte it would still compile due to the nature of compound assignment operators, but it would be performing a cast on each iteration - at least in the IL. It's quite possible that the JIT would take care of this, but there's no need to even ask it to :)
Fastest way would probably be a 256-element lookup table...
int[] lut
{
/*0x00*/ 0,
/*0x01*/ 1,
/*0x02*/ 1,
/*0x03*/ 2
...
/*0xFE*/ 7,
/*0xFF*/ 8
}
e.g.
11110000^01010101 = 10100101 -> lut[165] == 4
This is more commonly referred to as bit counting. There are literally dozens of different algorithms for doing this. Here is one site which lists a few of the more well known methods. There are even CPU specific instructions for doing this.
Theorectically, Microsoft could add a BitArray.CountSetBits function that gets JITed with the best algorithm for that CPU architecture. I, for one, would welcome such an addition.
As I understood it you want to sum the bits of each XOR between the left and right bytes.
for (int b = 0; b < left.Length; b++) {
int num = left[b] ^ right[b];
int sum = 0;
for (int i = 0; i < 8; i++) {
sum += (num >> i) & 1;
}
// do something with sum maybe?
}
I'm not sure if you mean sum the bytes or the bits.
To sum the bits within a byte, this should work:
int nSum = 0;
for (int i=0; i<=7; i++)
{
nSum += (byte_val>>i) & 1;
}
You would then need the xoring, and array looping around this, of course.
The following should do
int BitXorAndSum(byte[] left, byte[] right) {
int sum = 0;
for ( var i = 0; i < left.Length; i++) {
sum += SumBits((byte)(left[i] ^ right[i]));
}
return sum;
}
int SumBits(byte b) {
var sum = 0;
for (var i = 0; i < 8; i++) {
sum += (0x1) & (b >> i);
}
return sum;
}
It can be rewritten as ulong and use unsafe pointer, but byte is easier to understand:
static int BitCount(byte num)
{
// 0x5 = 0101 (bit) 0x55 = 01010101
// 0x3 = 0011 (bit) 0x33 = 00110011
// 0xF = 1111 (bit) 0x0F = 00001111
uint count = num;
count = ((count >> 1) & 0x55) + (count & 0x55);
count = ((count >> 2) & 0x33) + (count & 0x33);
count = ((count >> 4) & 0xF0) + (count & 0x0F);
return (int)count;
}
A general function to count bits could look like:
int Count1(byte[] a)
{
int count = 0;
for (int i = 0; i < a.Length; i++)
{
byte b = a[i];
while (b != 0)
{
count++;
b = (byte)((int)b & (int)(b - 1));
}
}
return count;
}
The less 1-bits, the faster this works. It simply loops over each byte, and toggles the lowest 1 bit of that byte until the byte becomes 0. The castings are necessary so that the compiler stops complaining about the type widening and narrowing.
Your problem could then be solved by using this:
int Count1Xor(byte[] a1, byte[] a2)
{
int count = 0;
for (int i = 0; i < Math.Min(a1.Length, a2.Length); i++)
{
byte b = (byte)((int)a1[i] ^ (int)a2[i]);
while (b != 0)
{
count++;
b = (byte)((int)b & (int)(b - 1));
}
}
return count;
}
A lookup table should be the fastest, but if you want to do it without a lookup table, this will work for bytes in just 10 operations.
public static int BitCount(byte value) {
int v = value - ((value >> 1) & 0x55);
v = (v & 0x33) + ((v >> 2) & 0x33);
return ((v + (v >> 4) & 0x0F));
}
This is a byte version of the general bit counting function described at Sean Eron Anderson's bit fiddling site.