We Keep Coding

Removing leading 0s in a byte array

I have a byte array as follows -
byte[] arrByt = new byte[] { 0xF, 0xF, 0x11, 0x4 };
so in binary
arrByt = 00001111 00001111 00010001 000000100
Now I want to create a new byte array by removing leading 0s for each byte from arrByt
arrNewByt = 11111111 10001100 = { 0xFF, 0x8C };
I know that this can be done by converting the byte values into binary string values, removing the leading 0s, appending the values and converting back to byte values into the new array.
However this is a slow process for a large array.
Is there a faster way to achieve this (like logical operations, bit operations, or other efficient ways)?
Thanks.

This should do the job quite fast. At least only standard loops and operators. Give it a try, will also work for longer source arrays.
// source array of bytes
var arrByt = new byte[] {0xF, 0xF, 0x11, 0x4 };
// target array - first with the size of the source array
var targetArray = new byte[arrByt.Length];
// bit index in target array
// from left = byte 0, bit 7 = index 31; to the right = byte 4, bit 0 = index 0
var targetIdx = targetArray.Length * 8 - 1;
// go through all bytes of the source array from left to right
for (var i = 0; i < arrByt.Length; i++)
{
var startFound = false;
// go through all bits of the current byte from the highest to the lowest
for (var x = 7; x >= 0; x--)
{
// copy the bit if it is 1 or if there was already a 1 before in this byte
if (startFound || ((arrByt[i] >> x) & 1) == 1)
{
startFound = true;
// copy the bit from its position in the source array to its new position in the target array
targetArray[targetArray.Length - ((targetIdx / 8) + 1)] |= (byte) (((arrByt[i] >> x) & 1) << (targetIdx % 8));
// advance the bit + byte position in the target array one to the right
targetIdx--;
}
}
}
// resize the target array to only the bytes that were used above
Array.Resize(ref targetArray, (int)Math.Ceiling((targetArray.Length * 8 - (targetIdx + 1)) / 8d));
// write target array content to console
for (var i = 0; i < targetArray.Length; i++)
{
Console.Write($"{targetArray[i]:X} ");
}
// OUTPUT: FF 8C

If you are trying to find the location of the most-significant bit, you can do a log2() of the byte (and if you don't have log2, you can use log(x)/log(2) which is the same as log2(x))
For instance, the number 7, 6, 5, and 4 all have a '1' in the 3rd bit position (0111, 0110, 0101, 0100). The log2() of them are all between 2 and 2.8. Same thing happens for anything in the 4th bit, it will be a number between 3 and 3.9. So you can find out the Most Significant Bit by adding 1 to the log2() of the number (round down).
floor(log2(00001111)) + 1 == floor(3.9) + 1 == 3 + 1 == 4
You know how many bits are in a byte, so you can easily know the number of bits to shift left:
int numToShift = 8 - floor(log2(bytearray[0])) + 1;
shiftedValue = bytearray[0] << numToShift;
From there, it's just a matter of keeping track of how many outstanding bits (not pushed into a bytearray yet) you have, and then pushing some/all of them on.
The above code would only work for the first byte array. If you put this in a loop, the numToShift would maybe need to keep track of the latest empty slot to shift things into (you might have to shift right to fit in current byte array, and then use the leftovers to put into the start of the next byte array). So instead of doing "8 -" in the above code, you would maybe put the starting location. For instance, if only 3 bits were left to fill in the current byte array, you would do:
int numToShift = 3 - floor(log2(bytearray[0])) + 1;
So that number should be a variable:
int numToShift = bitsAvailableInCurrentByte - floor(log2(bytearray[0])) + 1;

Please check this code snippet. This might help you.
byte[] arrByt = new byte[] { 0xF, 0xF, 0x11, 0x4 };
byte[] result = new byte[arrByt.Length / 2];
var en = arrByt.GetEnumerator();
int count = 0;
byte result1 = 0;
int index = 0;
while (en.MoveNext())
{
count++;
byte item = (byte)en.Current;
if (count == 1)
{
while (item < 128)
{
item = (byte)(item << 1);
}
result1 ^= item;
}
if (count == 2)
{
count = 0;
result1 ^= item;
result[index] = result1;
index++;
result1 = 0;
}
}
foreach (var s in result)
{
Console.WriteLine(s.ToString("X"));
}

C# - Padding image bytes with white bytes to fill 512 x 512

I'm using Digital Persona SDK to scan fingerprints in wsq format, for requeriment I need 512 x 512 image, the SDK only export 357 x 392 image.
The sdk provide a method to compress captured image from device in wsq format and return a byte array that I can write to disk.
-I've tried to allocate a buffer of 262144 for 512 x 512 image.
-Fill the new buffer with white pixel data each byte to value 255.
-Copy the original image buffer into the new image buffer. The original image doesn’t need to be centered but it's important to make sure to copy without corrupting the image data.
To summarize I've tried to copy the old image into the upper right corner of the new image.
DPUruNet.Compression.Start();
DPUruNet.Compression.SetWsqBitrate(95, 0);
Fid capturedImage = captureResult.Data;
//Fill the new buffer with white pixel data each byte to value 255.
byte[] bytesWSQ512 = new byte[262144];
for (int i = 0; i < bytesWSQ512.Length; i++)
{
bytesWSQ512[i] = 255;
}
//Compress capturedImage and get bytes (357 x 392)
byte[] bytesWSQ = DPUruNet.Compression.CompressRaw(capturedImage.Views[0].Width, capturedImage.Views[0].Height, 500, 8, capturedImage.Views[0].RawImage, CompressionAlgorithm.COMPRESSION_WSQ_NIST);
//Copy the original image buffer into the new image buffer
for (int i = 0; i < capturedImage.Views[0].Height; i++)
{
for (int j = 0; j < capturedImage.Views[0].Width; j++)
{
bytesWSQ512[i * bytesWSQ512.Length + j ] = bytesWSQ[i * capturedImage.Views[0].Width + j];
}
}
//Write bytes to disk
File.WriteAllBytes(#"C:\Users\Admin\Desktop\bytesWSQ512.wsq", bytesWSQ512);
DPUruNet.Compression.Finish();
When running that snippet I get IndexOutOfRangeException, I don't know if the loop or the calculation of indexes for new array are right.
Here is a representation of what I'm trying to do.

If someone is trying to achieve something like this or padding a raw image, I hope this will help.
DPUruNet.Compression.
DPUruNet.Compression.SetWsqBitrate(75, 0);
Fid ISOFid = captureResult.Data;
byte[] paddedImage = PadImage8BPP(captureResult.Data.Views[0].RawImage, captureResult.Data.Views[0].Width, captureResult.Data.Views[0].Height, 512, 512, 255);
byte[] bytesWSQ512 = Compression.CompressRaw(512, 512, 500, 8, paddedImage, CompressionAlgorithm.COMPRESSION_WSQ_NIST);
And the method to resize (pad) the image is:
public byte[] PadImage8BPP(byte[] original, int original_width, int original_height, int desired_width, int desired_height, byte pad_color)
{
byte[] canvas_8bpp = new byte[desired_width * desired_height];
for (int i = 0; i < canvas_8bpp.Length; i++)
canvas_8bpp[i] = pad_color; //Fill background. Note this type of fill will fail histogram checks.
int clamp_y_begin = 0;
int clamp_y_end = original_height;
int clamp_x_begin = 0;
int clamp_x_end = original_width;
int pad_y = 0;
int pad_x = 0;
if (original_height > desired_height)
{
int crop_distance = (int)Math.Ceiling((original_height - desired_height) / 2.0);
clamp_y_begin = crop_distance;
clamp_y_end = original_height - crop_distance;
}
else
{
pad_y = (desired_height - original_height) / 2;
}
if (original_width > desired_width)
{
int crop_distance = (int)Math.Ceiling((original_width - desired_width) / 2.0);
clamp_x_begin = crop_distance;
clamp_x_end = original_width - crop_distance;
}
else
{
pad_x = (desired_width - original_width) / 2;
}
//We traverse the captured image (either whole image or subset)
for (int y = clamp_y_begin; y < clamp_y_end; y++)
{
for (int x = clamp_x_begin; x < clamp_x_end; x++)
{
byte image_pixel = original[y * original_width + x];
canvas_8bpp[(pad_y + y - clamp_y_begin) * desired_width + pad_x + x - clamp_x_begin] = image_pixel;
}
}
return canvas_8bpp;
}

Convert byte[] to UInt16.

I have a 2d array of UInt16s which I've converted to raw bytes - I would like to take those bytes and convert them back into the original 2D array. I've managed to do this with a 2d array of doubles, but I can't figure out how to do it with UInt16.
Here's my code:
UInt16[,] dataArray;
//This array is populated with this data:
[4 6 2]
[0 2 0]
[1 3 4]
long byteCountUInt16Array = dataArray.GetLength(0) * dataArray.GetLength(1) * sizeof(UInt16);
var bufferUInt16 = new byte[byteCountUInt16Array];
Buffer.BlockCopy(newUint16Array, 0, bufferUInt16, 0, bufferUInt16.Length);
//Here is where I try to convert the values and print them out to see if the values are still the same:
UInt16[] originalUInt16Values = new UInt16[bufferUInt16.Length / 8];
for (int i = 0; i < 5; i++)
{
originalUInt16Values[i] = BitConverter.ToUInt16(bufferUInt16, i * 8);
Console.WriteLine("Values: " + originalUInt16Values[i]);
}
The print statement does not show the same values as the original 2d array. I'm pretty new to coding with bytes and UInt16 so most of this I'm learning in the process.
*Also, I know the last chunk of my code isn't putting values into a 2d array like the original array - right now I'm just trying to print out the values to see if they even match the original data.

If what you want is just to cast UInt16[,]->Byte, and then Byte->UInt16 you can do another Block copy, which is very fast at run-time, code should look like this:
UInt16[,] dataArray = new UInt16[,] {
{4, 6, 2},
{0, 2, 0},
{1, 3, 4}
};
for (int j = 0; j < 3; j++)
{
for (int i = 0; i < 3; i++)
{
Console.WriteLine("Value[" + i + ", " + j + "] = " + dataArray[j,i]);
}
}
long byteCountUInt16Array = dataArray.GetLength(0) * dataArray.GetLength(1) * sizeof(UInt16);
var bufferUInt16 = new byte[byteCountUInt16Array];
Buffer.BlockCopy(dataArray, 0, bufferUInt16, 0, bufferUInt16.Length);
//Here is where I try to convert the values and print them out to see if the values are still the same:
UInt16[] originalUInt16Values = new UInt16[bufferUInt16.Length / 2];
Buffer.BlockCopy(bufferUInt16, 0, originalUInt16Values, 0, BufferUInt16.Length);
for (int i = 0; i < 5; i++)
{
//originalUInt16Values[i] = BitConverter.ToUInt16(bufferUInt16, i * 8);
Console.WriteLine("Values---: " + originalUInt16Values[i]);
}
by the way, you only divided each UInt16 into two bytes, so you should calculate your new size dividing by two, not eight

The program
public static void Main(string[] args)
{
UInt16[,] dataArray = new ushort[,]{ {4,6,2}, {0,2,0}, {1,3,4}};
//This array is populated with this data:
long byteCountUInt16Array = dataArray.GetLength(0) * dataArray.GetLength(1) * sizeof(UInt16);
var byteBuffer = new byte[byteCountUInt16Array];
Buffer.BlockCopy(dataArray, 0, byteBuffer, 0, byteBuffer.Length);
for(int i=0; i < byteBuffer.Length; i++) {
Console.WriteLine("byteBuf[{0}]= {1}", i, byteBuffer[i]);
}
Console.WriteLine("Byte buffer len: {0} data array len: {1}", byteBuffer.Length, dataArray.GetLength(0)* dataArray.GetLength(1));
UInt16[] originalUInt16Values = new UInt16[byteBuffer.Length / 2];
for (int i = 0; i < byteBuffer.Length; i+=2)
{
ushort _a = (ushort)( (byteBuffer[i]) | (byteBuffer[i+1]) << 8);
originalUInt16Values[i/2] = _a;
Console.WriteLine("Values: " + originalUInt16Values[i/2]);
}
}
Outputs
byteBuf[0]= 4
byteBuf[1]= 0
byteBuf[2]= 6
byteBuf[3]= 0
byteBuf[4]= 2
byteBuf[5]= 0
byteBuf[6]= 0
byteBuf[7]= 0
byteBuf[8]= 2
byteBuf[9]= 0
byteBuf[10]= 0
byteBuf[11]= 0
byteBuf[12]= 1
byteBuf[13]= 0
byteBuf[14]= 3
byteBuf[15]= 0
byteBuf[16]= 4
byteBuf[17]= 0
Byte buffer len: 18 data array len: 9
Values: 4
Values: 6
Values: 2
Values: 0
Values: 2
Values: 0
Values: 1
Values: 3
Values: 4
You see that a ushort, aka UInt16 is stored in a byte-order in which 4 = 0x04 0x00, which is why I chose the conversion formula
ushort _a = (ushort)( (byteBuffer[i]) | (byteBuffer[i+1]) << 8);
Which will grab the byte at index i and take the next byte at i+1 and left shift it by the size of a byte (8 bits) to make up the 16 bits of a ushort. In orhter words, ushort _a = 0x[second byte] 0x[first byte], which is then repeated. This conversion code is specific for the endianess of the machine you are on and thus non-portable.
Also I fixed the error where the byteBuffer array was to big because it was multiplied with factor 8. A ushort is double the size of a byte, thus we only need factor 2 in the array length.

Addressing the title of your question (Convert byte[] to UInt16):
UInt16 result = (UInt16)BitConverter.ToInt16(yourByteArray, startIndex = 0);

Your casting up so you should be able to do things implicitly
var list = new List<byte> { 1, 2 ,
var uintList = new List<UInt16>();
//Cast in your select
uintList = list.Select(x => (UInt16)x).ToList();

Integer.parseInt(String s, int radix) in C#

I have this function in a Java program.
private static byte[] converToByte(String s)
{
byte[] output = new byte[s.length() / 2];
for (int i = 0, j = 0; i < s.length(); i += 2, j++)
{
output[j] = (byte)(Integer.parseInt(s.substring(i, i + 2), 16));
}
return output;
}
I am trying to create the same thing with C# but I'm having troubles. I tried this:
output[j] = (byte)(Int16.Parse(str.Substring(i, i + 2)));
But after a couple of iterations I got a System.OverflowException, what would be the instruction in C#?
Thanks.

private static sbyte[] converToByte(string s)
{
sbyte[] output = new sbyte[s.Length / 2];
for (int i = 0, j = 0; i < s.Length; i += 2, j++)
{
output[j] = (sbyte)(Convert.ToInt32(s.Substring(i, 2), 16));
}
return output;
}

You are using the wrong data type in your line:
output[j] = (byte)(Int16.Parse(str.Substring(i, i + 2)));
Short Name .NET Class Type Width Range (bits)
byte Byte Unsigned integer 8 0 to 255
short Int16 Signed integer 16 -32,768 to 32,767
You are getting an overflow exception because an Int16 (short) is far to big to fit into a byte.

After struggling with tihs problem myself I realised the real problem is that Java's substring method is:
substring(int beginIndex, int endIndex)
While C#'s implementation takes:
substring(int beginIndex, int length)
This means in the C# the same code is grabbing larger chunks of bytes causing an overflow.
#Dave Doknjas was on the right track but you can still convert to a byte with the new smaller chunk size.
output[j] = Convert.ToByte(str.Substring(i, i + 2), 16);

How do I convert an array of floats to a byte[] and back?

I have an array of Floats that need to be converted to a byte array and back to a float[]... can anyone help me do this correctly?
I'm working with the bitConverter class and found myself stuck trying to append the results.
The reason I'm doing this is so I can save runtime values into a IO Stream. The target storage is Azure Page blobs in case that matters. I don't care about what endian this is stored in, as long as it input matches the output.
static byte[] ConvertFloatToByteArray(float[] floats)
{
byte[] ret = new byte[floats.Length * 4];// a single float is 4 bytes/32 bits
for (int i = 0; i < floats.Length; i++)
{
// todo: stuck...I need to append the results to an offset of ret
ret = BitConverter.GetBytes(floats[i]);
}
return ret;
}
static float[] ConvertByteArrayToFloat(byte[] bytes)
{ //to do }

If you're looking for performance then you could use Buffer.BlockCopy. Nice and simple, and probably about as fast as you'll get in managed code.
var floatArray1 = new float[] { 123.45f, 123f, 45f, 1.2f, 34.5f };
// create a byte array and copy the floats into it...
var byteArray = new byte[floatArray1.Length * 4];
Buffer.BlockCopy(floatArray1, 0, byteArray, 0, byteArray.Length);
// create a second float array and copy the bytes into it...
var floatArray2 = new float[byteArray.Length / 4];
Buffer.BlockCopy(byteArray, 0, floatArray2, 0, byteArray.Length);
// do we have the same sequence of floats that we started with?
Console.WriteLine(floatArray1.SequenceEqual(floatArray2)); // True

There's the BitConverter.ToSingle(byte[] value, int startIndex) method that should help out here.
Returns a single-precision floating
point number converted from four bytes
at a specified position in a byte
array.
Your probably want something like (untested):
static float[] ConvertByteArrayToFloat(byte[] bytes)
{
if(bytes == null)
throw new ArgumentNullException("bytes");
if(bytes.Length % 4 != 0)
throw new ArgumentException
("bytes does not represent a sequence of floats");
return Enumerable.Range(0, bytes.Length / 4)
.Select(i => BitConverter.ToSingle(bytes, i * 4))
.ToArray();
}
EDIT: Non-LINQ:
float[] floats = new float[bytes.Length / 4];
for (int i = 0; i < bytes.Length / 4; i++)
floats[i] = BitConverter.ToSingle(bytes, i * 4);
return floats;

You are not moving the position when you copy the float[i] into the byte array, you should write something like
Array.Copy(BitConverter.GetBytes(float[i]),0,res,i*4);
instead of just:
ret = BitConverter.GetBytes(floats[i]);
the inverse function follow the same strategy.

static float[] ConvertByteArrayToFloat(byte[] bytes)
{
if(bytes.Length % 4 != 0) throw new ArgumentException();
float[] floats = new float[bytes.Length/4];
for(int i = 0; i < floats.Length; i++)
{
floats[i] = BitConverter.ToSingle(bytes, i*4);
}
return floats;
}