We Keep Coding

Removing leading 0s in a byte array

I have a byte array as follows -
byte[] arrByt = new byte[] { 0xF, 0xF, 0x11, 0x4 };
so in binary
arrByt = 00001111 00001111 00010001 000000100
Now I want to create a new byte array by removing leading 0s for each byte from arrByt
arrNewByt = 11111111 10001100 = { 0xFF, 0x8C };
I know that this can be done by converting the byte values into binary string values, removing the leading 0s, appending the values and converting back to byte values into the new array.
However this is a slow process for a large array.
Is there a faster way to achieve this (like logical operations, bit operations, or other efficient ways)?
Thanks.

This should do the job quite fast. At least only standard loops and operators. Give it a try, will also work for longer source arrays.
// source array of bytes
var arrByt = new byte[] {0xF, 0xF, 0x11, 0x4 };
// target array - first with the size of the source array
var targetArray = new byte[arrByt.Length];
// bit index in target array
// from left = byte 0, bit 7 = index 31; to the right = byte 4, bit 0 = index 0
var targetIdx = targetArray.Length * 8 - 1;
// go through all bytes of the source array from left to right
for (var i = 0; i < arrByt.Length; i++)
{
var startFound = false;
// go through all bits of the current byte from the highest to the lowest
for (var x = 7; x >= 0; x--)
{
// copy the bit if it is 1 or if there was already a 1 before in this byte
if (startFound || ((arrByt[i] >> x) & 1) == 1)
{
startFound = true;
// copy the bit from its position in the source array to its new position in the target array
targetArray[targetArray.Length - ((targetIdx / 8) + 1)] |= (byte) (((arrByt[i] >> x) & 1) << (targetIdx % 8));
// advance the bit + byte position in the target array one to the right
targetIdx--;
}
}
}
// resize the target array to only the bytes that were used above
Array.Resize(ref targetArray, (int)Math.Ceiling((targetArray.Length * 8 - (targetIdx + 1)) / 8d));
// write target array content to console
for (var i = 0; i < targetArray.Length; i++)
{
Console.Write($"{targetArray[i]:X} ");
}
// OUTPUT: FF 8C

If you are trying to find the location of the most-significant bit, you can do a log2() of the byte (and if you don't have log2, you can use log(x)/log(2) which is the same as log2(x))
For instance, the number 7, 6, 5, and 4 all have a '1' in the 3rd bit position (0111, 0110, 0101, 0100). The log2() of them are all between 2 and 2.8. Same thing happens for anything in the 4th bit, it will be a number between 3 and 3.9. So you can find out the Most Significant Bit by adding 1 to the log2() of the number (round down).
floor(log2(00001111)) + 1 == floor(3.9) + 1 == 3 + 1 == 4
You know how many bits are in a byte, so you can easily know the number of bits to shift left:
int numToShift = 8 - floor(log2(bytearray[0])) + 1;
shiftedValue = bytearray[0] << numToShift;
From there, it's just a matter of keeping track of how many outstanding bits (not pushed into a bytearray yet) you have, and then pushing some/all of them on.
The above code would only work for the first byte array. If you put this in a loop, the numToShift would maybe need to keep track of the latest empty slot to shift things into (you might have to shift right to fit in current byte array, and then use the leftovers to put into the start of the next byte array). So instead of doing "8 -" in the above code, you would maybe put the starting location. For instance, if only 3 bits were left to fill in the current byte array, you would do:
int numToShift = 3 - floor(log2(bytearray[0])) + 1;
So that number should be a variable:
int numToShift = bitsAvailableInCurrentByte - floor(log2(bytearray[0])) + 1;

Please check this code snippet. This might help you.
byte[] arrByt = new byte[] { 0xF, 0xF, 0x11, 0x4 };
byte[] result = new byte[arrByt.Length / 2];
var en = arrByt.GetEnumerator();
int count = 0;
byte result1 = 0;
int index = 0;
while (en.MoveNext())
{
count++;
byte item = (byte)en.Current;
if (count == 1)
{
while (item < 128)
{
item = (byte)(item << 1);
}
result1 ^= item;
}
if (count == 2)
{
count = 0;
result1 ^= item;
result[index] = result1;
index++;
result1 = 0;
}
}
foreach (var s in result)
{
Console.WriteLine(s.ToString("X"));
}

Reading 32-bit grayscale Tiff using Libtiff.Net

I've tried to read a 32-bit grayscale tiff file which each pixel in the image contains a floating point number. But during the reading process, the buffer array contains 4 values for each pixel. For instance [ pixel value = 43.0 --> byte values for the pixel = {0 , 0 , 44 , 66}]. I can't understand the relation between float pixel value and the byte values. I also wrote the image using the buffer but pixel values for output image are int values like 1073872896. Any suggestion would be appreciated.
using (Tiff input = Tiff.Open(#"E:\Sample_04.tif", "r"))
{
// get properties to use in writing output image file
int width = input.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int height = input.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
int samplesPerPixel = input.GetField(TiffTag.SAMPLESPERPIXEL)[0].ToInt();
int bitsPerSample = input.GetField(TiffTag.BITSPERSAMPLE)[0].ToInt();
int photo = input.GetField(TiffTag.PHOTOMETRIC)[0].ToInt();
int scanlineSize = input.ScanlineSize();
byte[][] buffer = new byte[height][];
for (int i = 0; i < height; i++)
{
buffer[i] = new byte[scanlineSize];
input.ReadScanline(buffer[i], i);
}
using (Tiff output = Tiff.Open("output.tif", "w"))
{
output.SetField(TiffTag.SAMPLESPERPIXEL, samplesPerPixel);
output.SetField(TiffTag.IMAGEWIDTH, width);
output.SetField(TiffTag.IMAGELENGTH, height);
output.SetField(TiffTag.BITSPERSAMPLE, bitsPerSample);
output.SetField(TiffTag.ROWSPERSTRIP, output.DefaultStripSize(0));
output.SetField(TiffTag.PHOTOMETRIC, photo);
output.SetField(TiffTag.PLANARCONFIG, PlanarConfig.CONTIG);
output.SetField(TiffTag.COMPRESSION, compression);
int j = 0;
for (int i = 0; i < h; i++)
{
output.WriteScanline(buffer[i], j);
j++;
}
}
}
Update 1:
I found out the relation between four bytes and the pixel value using BitConverter class in c# that is like this:
byte[] a = { 0, 0, 44, 66 } --> 43 = BitConverter.ToSingle(a, 0) and 1110179840 = BitConverter.ToInt32(a, 0). It seem bytes are converted to int32 and now the question is how convert byte values to float?
Update 2:
The original tiff file and the tiff after writing the snippet code have been attached.Why is the output tiff file messed up?

I added this line of code to convert pixel values to floating number and it works fine.
output.SetField(TiffTag.SAMPLEFORMAT, SampleFormat.IEEEFP);

C# How to send colorDialog value to a textBox in 5bitRGB?

So I'm creating an editor of a PS2 game.
And this game has two "Systems" of colors.
The "normal" RGB R: 0 to 255 G: 0 to 255 B: 0 to 255.
And the or I think it is 5bitRGB R: 0 to 31 G: 0 to 31 B: 0 to 31.
And to make the color change in the game,
I have to convert the chosen values in the
colorDialog in Hexadecimal for example: R: 255 G: 176 B: 15
In Hexadecimal stands FFB00F.
And then later change these values in the "slots" of 3 bytes via Hex.
Beauty so far so good, but 5bitRGB only have "slots" of 2 bytes.
Example: 5bitRGB R: 31 G: 0 B: 0 in Hex 1F80.
And that's where I do not know what to do, because the colors of the normal RGB
I can send the values in Hexadecimal to the textBox.
And then I saved these values textBox in "slots" of 3 bytes via Hex.
Meanwhile the slots for the 5bitRGB color change via Hex
They are only "slots" of 2 bytes.
So I would have to send the converted colorDialog value to 5bitRGB
for textBox in 2 bytes, is this really possible?

So I would have to send the converted colorDialog value to 5bitRGB for textBox in 2 bytes, is this really possible?
Sure! 5-bits x 3 fields = 15-bits, and 2 bytes = 16-bits. You'll even have room left over, but not much.
It sounds like it's just a matter of handling a reduction in the resolution of each field. This can be done with a bitwise shift operation to reduce each 8-bit field to a 5-bit field.
You'll need to right shift the value for storage as a 5-bit field. You can also left shift the value to use as an 8-bit (byte) field for setting the Color property - but note that this is just an 8-bit representation of a 5-bit value and the loss of resolution from shifting to 5-bits will persist. You will have to decide on a convention to handle the loss of resolution - doing nothing is an option.
For example:
// 8-bit color values
byte rValue = 255;
byte gValue = 127;
byte bValue = 63;
// set color
pictureBox1.BackColor = Color.FromArgb(rValue, gValue, bValue);
// 5-bit color values
var rBits = new BitArray(5, false);
var gBits = new BitArray(5, false);
var bBits = new BitArray(5, false);
// bit position comparison operator
byte op = 0x80;
// convert to 5-bit arrays
for (int i = 0; i < 5; i++)
{
if (rValue > op) { rBits[i] = true; rValue -= op; }
if (gValue > op) { gBits[i] = true; gValue -= op; }
if (bValue > op) { bBits[i] = true; bValue -= op; }
op >>= 1;
}
byte rRetrieved = 0;
byte gRetrieved = 0;
byte bRetrieved = 0;
// bit position comparison operator
op = 0x80;
// convert back to 8-bit bytes
for (int i = 0; i < 5; i++)
{
if (rBits[i]) { rRetrieved += op; }
if (gBits[i]) { gRetrieved += op; }
if (bBits[i]) { bRetrieved += op; }
op >>= 1;
}
// set color - note loss of resolution due to shifting
pictureBox1.BackColor = Color.FromArgb(rRetrieved, gRetrieved, bRetrieved);

Getting PCM values of WAV files

I have a .wav mono file (16bit,44.1kHz) and im using this code below. If im not wrong, this would give me an output of values between -1 and 1 which i can apply FFT on ( to be converted to a spectrogram later on). However, my output is no where near -1 and 1.
This is a portion of my output
7.01214599609375
17750.2552337646
8308.42733764648
0.000274658203125
1.00001525878906
0.67291259765625
1.3458251953125
16.0000305175781
24932
758.380676269531
0.0001068115234375
This is the code which i got from another post
Edit 1:
public static Double[] prepare(String wavePath, out int SampleRate)
{
Double[] data;
byte[] wave;
byte[] sR = new byte[4];
System.IO.FileStream WaveFile = System.IO.File.OpenRead(wavePath);
wave = new byte[WaveFile.Length];
data = new Double[(wave.Length - 44) / 4];//shifting the headers out of the PCM data;
WaveFile.Read(wave, 0, Convert.ToInt32(WaveFile.Length));//read the wave file into the wave variable
/***********Converting and PCM accounting***************/
for (int i = 0; i < data.Length; i += 2)
{
data[i] = BitConverter.ToInt16(wave, i) / 32768.0;
}
/**************assigning sample rate**********************/
for (int i = 24; i < 28; i++)
{
sR[i - 24] = wave[i];
}
SampleRate = BitConverter.ToInt16(sR, 0);
return data;
}
Edit 2 : Im getting ouput with 0s every 2nd number
0.009002685546875
0
0.009613037109375
0
0.0101318359375
0
0.01080322265625
0
0.01190185546875
0
0.01312255859375
0
0.014068603515625

If your samples are 16 bits (which appears to be the case), then you want to work with Int16. Each 2 bytes of the sample data is a signed 16-bit integer in the range -32768 .. 32767, inclusive.
If you want to convert a signed Int16 to a floating point value from -1 to 1, then you have to divide by Int16.MaxValue + 1 (which is equal to 32768). So, your code becomes:
for (int i = 0; i < data.Length; i += 2)
{
data[i] = BitConverter.ToInt16(wave, i) / 32768.0;
}
We use 32768 here because the values are signed.
So -32768/32768 will give -1.0, and 32767/32768 gives 0.999969482421875.
If you used 65536.0, then your values would only be in the range -0.5 .. 0.5.

Inserting bytes in the middle of binary file

I want to add some string in the middle of image metadata block. Under some specific marker. I have to do it on bytes level since .NET has no support for custom metadata fields.
The block is built like 1C 02 XX YY YY ZZ ZZ ZZ ... where XX is the ID of the field I need to append and YY YY is the size of it, ZZ = data.
I imagine it should be more or less possible to read all the image data up to this marker (1C 02 XX) then increase the size bytes (YY YY), add data at the end of ZZ and then add the rest of the original file? Is this correct?
How should I go on with it? It needs to work as fast as possible with 4-5 MB JPEG files.

In general there is no way to speed up this operation. You have to read at least portion that needs to be moved and write it again in updated file. Creating new file and copying content to it may be faster if you can parallelize read and write operations.
Note: In you particular case it may not be possible to just insert content in the middle of the file as most of file formats are not designed with such modifcations in mind. Often there are offsets to portions of the file that will be invalid when you shift part of the file. Specifying what file format you trying to work with may help other people to provide better approaches.

Solved the problem with this code:
List<byte> dataNew = new List<byte>();
byte[] data = File.ReadAllBytes(jpegFilePath);
int j = 0;
for (int i = 1; i < data.Length; i++)
{
if (data[i - 1] == (byte)0x1C) // 1C IPTC
{
if (data[i] == (byte)0x02) // 02 IPTC
{
if (data[i + 1] == (byte)fileByte) // IPTC field_number, i.e. 0x78 = IPTC_120
{
j = i;
break;
}
}
}
}
for (int i = 0; i < j + 2; i++) // add data from file before this field
dataNew.Add(data[i]);
int countOld = (data[j + 2] & 255) << 8 | (data[j + 3] & 255); // curr field length
int countNew = valueToAdd.Length; // new string length
int newfullSize = countOld + countNew; // sum
byte[] newSize = BitConverter.GetBytes((Int16)newfullSize); // Int16 on 2 bytes (to use 2 bytes as size)
Array.Reverse(newSize); // changes order 10 00 to 00 10
for (int i = 0; i < newSize.Length; i++) // add changed size
dataNew.Add(newSize[i]);
for (int i = j + 4; i < j + 4 + countOld; i++) // add old field value
dataNew.Add(data[i]);
byte[] newString = ASCIIEncoding.ASCII.GetBytes(valueToAdd);
for (int i = 0; i < newString.Length; i++) // append with new field value
dataNew.Add(newString[i]);
for (int i = j + 4 + newfullSize; i < data.Length; i++) // add rest of the file
dataNew.Add(data[i]);
byte[] finalArray = dataNew.ToArray();
File.WriteAllBytes(Path.Combine(Path.GetDirectoryName(jpegFilePath), "newfile.jpg"), finalArray);

Here is an easy and quite fast solution. It moves all bytes after given offset to their new position according to given extraBytes, so you can insert your data.
public void ExpandFile(FileStream stream, long offset, int extraBytes)
{
// http://stackoverflow.com/questions/3033771/file-io-with-streams-best-memory-buffer-size
const int SIZE = 4096;
var buffer = new byte[SIZE];
var length = stream.Length;
// Expand file
stream.SetLength(length + extraBytes);
var pos = length;
int to_read;
while (pos > offset)
{
to_read = pos - SIZE >= offset ? SIZE : (int)(pos - offset);
pos -= to_read;
stream.Position = pos;
stream.Read(buffer, 0, to_read);
stream.Position = pos + extraBytes;
stream.Write(buffer, 0, to_read);
}
Need to be checked, though...