We Keep Coding

How to increase volume/amplitude on raw audio bytes

I'm working with phone raw phone sounds and recordings and I want to normalize them to a certain volume level in a .Net C# project.
The sound is a collection of raw audio bytes (mono unheadered 16-bit signed PCM audio 16000Hz).
The audio is split into blocks of 3200 bytes == 100ms.
Any suggestions how to increase the volume/amplitude so the sounds is louder?
I haven't got a clue if I need to add a constant or multiply values, or if I need to do it to every 1,2,3.... bytes? And maybe there is already a open source solution for this?

To answer my own question (for others).
The solution is to multiply every sample (when 16bit PCM that are 2 bytes) with a constant value.
Do avoid overflow\to much increase you can calculate the highest constant value you can use by looking for the highest sample value and calculate the multiply factor to get it to highest sample value possible, in 16bit PCM case thats 32676 or something.
Here is litle example:
public byte[] IncreaseDecibel(byte[] audioBuffer, float multiplier)
{
// Max range -32768 and 32767
var highestValue = GetHighestAbsoluteSample(audioBuffer);
var highestPosibleMultiplier = (float)Int16.MaxValue/highestValue; // Int16.MaxValue = 32767
if (multiplier > highestPosibleMultiplier)
{
multiplier = highestPosibleMultiplier;
}
for (var i = 0; i < audioBuffer.Length; i = i + 2)
{
Int16 sample = BitConverter.ToInt16(audioBuffer, i);
sample *= (Int16)(sample * multiplier);
byte[] sampleBytes = GetLittleEndianBytesFromShort(sample);
audioBuffer[i] = sampleBytes[sampleBytes.Length-2];
audioBuffer[i+1] = sampleBytes[sampleBytes.Length-1];
}
return audioBuffer;
}
// ADDED GetHighestAbsoluteSample, hopefully its still correct because code has changed over time
/// <summary>
/// Peak sample value
/// </summary>
/// <param name="audioBuffer">audio</param>
/// <returns>0 - 32768</returns>
public static short GetHighestAbsoluteSample(byte[] audioBuffer)
{
Int16 highestAbsoluteValue = 0;
for (var i = 0; i < (audioBuffer.Length-1); i = i + 2)
{
Int16 sample = ByteConverter.GetShortFromLittleEndianBytes(audioBuffer, i);
// prevent Math.Abs overflow exception
if (sample == Int16.MinValue)
{
sample += 1;
}
var absoluteValue = Math.Abs(sample);
if (absoluteValue > highestAbsoluteValue)
{
highestAbsoluteValue = absoluteValue;
}
}
return (highestAbsoluteValue > LowestPossibleAmplitude) ?
highestAbsoluteValue : LowestPossibleAmplitude;
}

Is this the correct way to truncate a hash in C#?

I have a requirement to hash input strings and produce 14 digit decimal numbers as output.
The math I am using tells me I can have, at maximum, a 46 bit unsigned integer.
I am aware that a 46 bit uint means less collision resistance for any potential hash function. However, the number of hashes I am creating keeps the collision probability in an acceptable range.
I would be most grateful if the community could help me verify that my method for truncating a hash to 46 bits is solid. I have a gut feeling that there are optimizations and/or easier ways to do this. My function is as follows (where bitLength is 46 when this function is called):
public static UInt64 GetTruncatedMd5Hash(string input, int bitLength)
{
var md5Hash = MD5.Create();
byte[] fullHashBytes = md5Hash.ComputeHash(Encoding.UTF8.GetBytes(input));
var fullHashBits = new BitArray(fullHashBytes);
// BitArray stores LSB of each byte in lowest indexes, so reversing...
ReverseBitArray(fullHashBits);
// truncate by copying only number of bits specified by bitLength param
var truncatedHashBits = new BitArray(bitLength);
for (int i = 0; i < bitLength - 1; i++)
{
truncatedHashBits[i] = fullHashBits[i];
}
byte[] truncatedHashBytes = new byte[8];
truncatedHashBits.CopyTo(truncatedHashBytes, 0);
return BitConverter.ToUInt64(truncatedHashBytes, 0);
}
Thanks for taking a look at this question. I appreciate any feedback!

With the help of the comments above, I crafted the following solution:
public static UInt64 GetTruncatedMd5Hash(string input, int bitLength)
{
if (string.IsNullOrWhiteSpace(input)) throw new ArgumentException("input must not be null or whitespace");
if(bitLength > 64) throw new ArgumentException("bitLength must be <= 64");
var md5Hash = MD5.Create();
byte[] fullHashBytes = md5Hash.ComputeHash(Encoding.UTF8.GetBytes(input));
if(bitLength == 64)
return BitConverter.ToUInt64(fullHashBytes, 0);
var bitMask = (1UL << bitLength) - 1UL;
return BitConverter.ToUInt64(fullHashBytes, 0) & bitMask;
}
It's much tighter (and faster) than what I was trying to do before.

How to set each bit in a byte array

How do I set each bit in the following byte array which has 21 bytes or 168 bits to either zero or one?
byte[] logonHours
Thank you very much

Well, to clear every bit to zero you can just use Array.Clear:
Array.Clear(logonHours, 0, logonHours.Length);
Setting each bit is slightly harder:
for (int i = 0; i < logonHours.Length; i++)
{
logonHours[i] = 0xff;
}
If you find yourself filling an array often, you could write an extension method:
public static void FillArray<T>(this T[] array, T value)
{
// TODO: Validation
for (int i = 0; i < array.Length; i++)
{
array[i] = value;
}
}

BitArray.SetAll:
System.Collections.BitArray a = new System.Collections.BitArray(logonHours);
a.SetAll(true);
Note that this copies the data from the byte array. It's not just a wrapper around it.

This may be more than you need, but ...
Usually when dealing with individual bits in any data type, I define a const for each bit position, then use the binary operators |, &, and ^.
i.e.
const byte bit1 = 1;
const byte bit2 = 2;
const byte bit3 = 4;
const byte bit4 = 8;
.
.
const byte bit8 = 128;
Then you can turn whatever bits you want on and off using the bit operations.
byte byTest = 0;
byTest = byTest | bit4;
would turn bit 4 on but leave the rest untouched.
You would use the & and ^ to turn them off or do more complex exercises.
Obviously, since you only want to turn all bits up or down then you can just set the byte to 0 or 255. That would turn them all off or on.

Variable length encoding of an integer

Whats the best way of doing variable length encoding of an unsigned integer value in C# ?
"The actual intent is to append a variable length encoded integer (bytes) to a file header."
For ex: "Content-Length" - Http Header
Can this be achieved with some changes in the logic below.
I have written some code which does that ....

A method I have used, which makes smaller values use fewer bytes, is to encode 7 bits of data + 1 bit of overhead pr. byte.
The encoding works only for positive values starting with zero, but can be modified if necessary to handle negative values as well.
The way the encoding works is like this:
Grab the lowest 7 bits of your value and store them in a byte, this is what you're going to output
Shift the value 7 bits to the right, getting rid of those 7 bits you just grabbed
If the value is non-zero (ie. after you shifted away 7 bits from it), set the high bit of the byte you're going to output before you output it
Output the byte
If the value is non-zero (ie. same check that resulted in setting the high bit), go back and repeat the steps from the start
To decode:
Start at bit-position 0
Read one byte from the file
Store whether the high bit is set, and mask it away
OR in the rest of the byte into your final value, at the bit-position you're at
If the high bit was set, increase the bit-position by 7, and repeat the steps, skipping the first one (don't reset the bit-position)
39 32 31 24 23 16 15 8 7 0
value: |DDDDDDDD|CCCCCCCC|BBBBBBBB|AAAAAAAA|
encoded: |0000DDDD|xDDDDCCC|xCCCCCBB|xBBBBBBA|xAAAAAAA| (note, stored in reverse order)
As you can see, the encoded value might occupy one additional byte that is just half-way used, due to the overhead of the control bits. If you expand this to a 64-bit value, the additional byte will be completely used, so there will still only be one byte of extra overhead.
Note: Since the encoding stores values one byte at a time, always in the same order, big- or little-endian systems will not change the layout of this. The least significant byte is always stored first, etc.
Ranges and their encoded size:
0 - 127 : 1 byte
128 - 16.383 : 2 bytes
16.384 - 2.097.151 : 3 bytes
2.097.152 - 268.435.455 : 4 bytes
268.435.456 - max-int32 : 5 bytes
Here's C# implementations for both:
void Main()
{
using (FileStream stream = new FileStream(#"c:\temp\test.dat", FileMode.Create))
using (BinaryWriter writer = new BinaryWriter(stream))
writer.EncodeInt32(123456789);
using (FileStream stream = new FileStream(#"c:\temp\test.dat", FileMode.Open))
using (BinaryReader reader = new BinaryReader(stream))
reader.DecodeInt32().Dump();
}
// Define other methods and classes here
public static class Extensions
{
/// <summary>
/// Encodes the specified <see cref="Int32"/> value with a variable number of
/// bytes, and writes the encoded bytes to the specified writer.
/// </summary>
/// <param name="writer">
/// The <see cref="BinaryWriter"/> to write the encoded value to.
/// </param>
/// <param name="value">
/// The <see cref="Int32"/> value to encode and write to the <paramref name="writer"/>.
/// </param>
/// <exception cref="ArgumentNullException">
/// <para><paramref name="writer"/> is <c>null</c>.</para>
/// </exception>
/// <exception cref="ArgumentOutOfRangeException">
/// <para><paramref name="value"/> is less than 0.</para>
/// </exception>
/// <remarks>
/// See <see cref="DecodeInt32"/> for how to decode the value back from
/// a <see cref="BinaryReader"/>.
/// </remarks>
public static void EncodeInt32(this BinaryWriter writer, int value)
{
if (writer == null)
throw new ArgumentNullException("writer");
if (value < 0)
throw new ArgumentOutOfRangeException("value", value, "value must be 0 or greater");
do
{
byte lower7bits = (byte)(value & 0x7f);
value >>= 7;
if (value > 0)
lower7bits |= 128;
writer.Write(lower7bits);
} while (value > 0);
}
/// <summary>
/// Decodes a <see cref="Int32"/> value from a variable number of
/// bytes, originally encoded with <see cref="EncodeInt32"/> from the specified reader.
/// </summary>
/// <param name="reader">
/// The <see cref="BinaryReader"/> to read the encoded value from.
/// </param>
/// <returns>
/// The decoded <see cref="Int32"/> value.
/// </returns>
/// <exception cref="ArgumentNullException">
/// <para><paramref name="reader"/> is <c>null</c>.</para>
/// </exception>
public static int DecodeInt32(this BinaryReader reader)
{
if (reader == null)
throw new ArgumentNullException("reader");
bool more = true;
int value = 0;
int shift = 0;
while (more)
{
byte lower7bits = reader.ReadByte();
more = (lower7bits & 128) != 0;
value |= (lower7bits & 0x7f) << shift;
shift += 7;
}
return value;
}
}

You should first make an histogram of your value. If the distribution is random (that is, every bin of your histogram's count is close to the other), then you'll not be able encode more efficiently than the binary representation for this number.
If your histogram is unbalanced (that is, if some values are more present than others), then it might make sense to choose an encoding that's using less bits for these values, while using more bits for the other -unlikely- values.
For example, if the number you need to encode are 2x more likely to be smaller than 15 bits than larger, you can use the 16-th bit to tell so and only store/send 16 bits (if it's zero, then the upcoming byte will form a 16-bits numbers that can fit in a 32 bits number).
If it's 1, then the upcoming 25 bits will form a 32 bits numbers.
You loose one bit here but because it's unlikely, in the end, for a lot of number, you win more bits.
Obviously, this is a trivial case, and the extension of this to more than 2 cases is the Huffman algorithm that affect a "code word" that close-to optimum based on the probability of the numbers to appear.
There's also the arithmetic coding algorithm that does this too (and probably other).
In all cases, there is no solution that can store random value more efficiently than what's being done currently in computer memory.
You have to think about how long and how hard will be the implementation of such solution compared to the saving you'll get in the end to know if it's worth it. The language itself is not relevant here.

If small values are more common than large ones you can use Golomb coding.

I know this question was asked quite a few years ago, however for MIDI developers I thought to share some code from a personal midi project I'm working on. The code block is based on a segment from the book Maximum MIDI by Paul Messick (This example is a tweaked version for my own needs however, the concept is all there...).
public struct VariableLength
{
// Variable Length byte array to int
public VariableLength(byte[] bytes)
{
int index = 0;
int value = 0;
byte b;
do
{
value = (value << 7) | ((b = bytes[index]) & 0x7F);
index++;
} while ((b & 0x80) != 0);
Length = index;
Value = value;
Bytes = new byte[Length];
Array.Copy(bytes, 0, Bytes, 0, Length);
}
// Variable Length int to byte array
public VariableLength(int value)
{
Value = value;
byte[] bytes = new byte[4];
int index = 0;
int buffer = value & 0x7F;
while ((value >>= 7) > 0)
{
buffer <<= 8;
buffer |= 0x80;
buffer += (value & 0x7F);
}
while (true)
{
bytes[index] = (byte)buffer;
index++;
if ((buffer & 0x80) > 0)
buffer >>= 8;
else
break;
}
Length = index;
Bytes = new byte[index];
Array.Copy(bytes, 0, Bytes, 0, Length);
}
// Number of bytes used to store the variable length value
public int Length { get; private set; }
// Variable Length Value
public int Value { get; private set; }
// Bytes representing the integer value
public byte[] Bytes { get; private set; }
}
How to use:
public void Example()
{
//Convert an integer into a variable length byte
int varLenVal = 480;
VariableLength v = new VariableLength(varLenVal);
byte[] bytes = v.Bytes;
//Convert a variable length byte array into an integer
byte[] varLenByte = new byte[2]{131, 96};
VariableLength v = new VariableLength(varLenByte);
int result = v.Length;
}

As Grimbly pointed out, there exists BinaryReader.Read7BitEncodedInt and BinaryWriter.Write7BitEncodedInt. However, these are internal methods that one cannot call from a BinaryReader or -Writer object.
However, what you can do is take the internal implementation and copy it from the reader and the writer:
public static int Read7BitEncodedInt(this BinaryReader br) {
// Read out an Int32 7 bits at a time. The high bit
// of the byte when on means to continue reading more bytes.
int count = 0;
int shift = 0;
byte b;
do {
// Check for a corrupted stream. Read a max of 5 bytes.
// In a future version, add a DataFormatException.
if (shift == 5 * 7) // 5 bytes max per Int32, shift += 7
throw new FormatException("Format_Bad7BitInt32");
// ReadByte handles end of stream cases for us.
b = br.ReadByte();
count |= (b & 0x7F) << shift;
shift += 7;
} while ((b & 0x80) != 0);
return count;
}
public static void Write7BitEncodedInt(this BinaryWriter br, int value) {
// Write out an int 7 bits at a time. The high bit of the byte,
// when on, tells reader to continue reading more bytes.
uint v = (uint)value; // support negative numbers
while (v >= 0x80) {
br.Write((byte)(v | 0x80));
v >>= 7;
}
br.Write((byte)v);
}
When you include this code in any class of your project, you'll be able to use the methods on any BinaryReader/BinaryWriter object. They've only been slightly modified to make them work outside of their original classes (for example by changing ReadByte() to br.ReadByte()). The comments are from the original source.

BinaryReader.Read7BitEncodedInt Method ?
BinaryWriter.Write7BitEncodedInt Method ?

C# get and set the high order word of an integer

What's an efficient or syntactically simple way to get and set the high order part of an integer?

There are multiple ways of achieving that, here are some of them.
Using the Bitwise and/or Shift operators
Applying a right shift in an integer will move the bits to the right, putting zeros to the left.
In the case below, it will shift the size of a short (Int16, as 16 bits).
Applying a logical AND (&) operation in an integer like 0x0000FFFF will basically 'cut' the value (where it's F) and ignore the rest (where it's 0).
Remember that in the end it's just a 0b_1 AND 0b_1 = 0b_1 operation, so any 0b_0 AND 0b_1 will result in 0b_0.
Applying a logical OR (|) operation will basically merge the two numbers in this case, like 0b_10 | 0b_01 = 0b_11.
Code:
uint number = 0xDEADBEEF;
//Get the higher order value.
var high = number >> 16;
Console.WriteLine($"High: {high:X}");
//Get the lower order value.
var low = number & 0xFFFF; //Or use 0x0000FFFF
Console.WriteLine($"Low: {low:X}");
//Set a high order value (you can also use 0xFFFF instead of 0x0000FFFF).
uint newHigh = 0xFADE;
number = number & 0x0000FFFF | newHigh << 16;
Console.WriteLine($"New high: {number:X}");
//Set a low order value.
uint newLow = 0xC0DE;
number = number & 0xFFFF0000 | newLow & 0x0000FFFF;
Console.WriteLine($"New low: {number:X}");
Output:
High: DEAD
Low: BEEF
New high: FADEBEEF
New low: FADEC0DE
Using FieldOffsetAttribute in a struct
C# has excellent support for variables sharing the same memory location, and bits structuring.
Since C# has no macro functions like in C, you can use the union approach to speed things up. It's more performant than passing the variable to methods or extension methods.
You can do that by simply creating a struct with explicit layout and setting the offset of the fields:
Code:
using System;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Explicit)]
struct WordUnion
{
[FieldOffset(0)]
public uint Number;
[FieldOffset(0)]
public ushort Low;
[FieldOffset(2)]
public ushort High;
}
public class MainClass
{
public static void Main(string[] args)
{
var x = new WordUnion { Number = 0xABADF00D };
Console.WriteLine("{0:X} {1:X} {2:X}", x.Number, x.High, x.Low);
x.Low = 0xFACE;
Console.WriteLine("{0:X} {1:X} {2:X}", x.Number, x.High, x.Low);
x.High = 0xDEAD;
Console.WriteLine("{0:X} {1:X} {2:X}", x.Number, x.High, x.Low);
}
}
Output:
ABADF00D ABAD F00D
ABADFACE ABAD FACE
DEADFACE DEAD FACE
Mind that with Visual Studio 2029 (16.7), you still may get zeros in x.High and x.Low when adding the variable x inside the Watch or by hovering your cursor on top of the variables x.High and x.Low directly.
Using unsafe and pointer element access operator []
To a more akin to C programming, but in C#, use unsafe:
Code:
unsafe
{
uint value = 0xCAFEFEED;
// x86 is using low-endian.
// So low order array number gets the low order of the value
// And high order array number gets the high order of the value
Console.WriteLine("Get low order of {0:X}: {1:X}",
value, ((ushort*) &value)[0]);
Console.WriteLine("Get high order of {0:X}: {1:X}",
value, ((ushort*) &value)[1]);
((ushort*) &value)[1] = 0xABAD;
Console.WriteLine("Set high order to ABAD: {0:X}", value);
((ushort*) &value)[0] = 0xFACE;
Console.WriteLine("Set low order to FACE: {0:X}", value);
}
Output:
Get low order of CAFEFEED: FEED
Get high order of CAFEFEED: CAFE
Set high order to ABAD: ABADFEED
Set low order to FACE: ABADFACE
Using unsafe and pointer member access operator ->
Another unsafe approach, but this time accessing a member from the WordUnion struct declared in a previous example:
Code:
unsafe
{
uint value = 0xCAFEFEED;
Console.WriteLine("Get low order of {0:X}: {1:X}",
value, ((WordUnion*) &value)->Low);
Console.WriteLine("Get high order of {0:X}: {1:X}",
value, ((WordUnion*) &value)->High);
((WordUnion*) &value)->High = 0xABAD;
Console.WriteLine($"Set high order to ABAD: {value:X}");
((WordUnion*) &value)->Low = 0xFACE;
Console.WriteLine($"Set low order to FACE: {value:X}");
}
Output:
Get low order of CAFEFEED: FEED
Get high order of CAFEFEED: CAFE
Set high order to ABAD: ABADFEED
Set low order to FACE: ABADFACE
Using the BitConverter class
It simply gets 16 bits (2 bytes, a short/Int16) from the specified number. The offset can be controlled by the second parameter.
Code:
uint value = 0xCAFEFEED;
var low = BitConverter.ToInt16(BitConverter.GetBytes(value), 0);
var high = BitConverter.ToInt16(BitConverter.GetBytes(value), 2);
Console.WriteLine($"Low: {low:X}");
Console.WriteLine($"High: {high:X}");
Output:
Low: 0xCAFE
High: 0xFEED

It's the same as in C/C++:
// get the high order 16 bits
int high = 0x12345678 >> 16; // high = 0x1234
// set the high order 16 bits
high = (high & 0x0000FFFF) + (0x5678 << 16); // high = 0x56781234
EDIT: Because I'm in a good mood, here you go. Just remember, immutable types are immutable! The 'set' functions need to be assigned to something.
public static class ExtensionMethods
{
public int LowWord(this int number)
{ return number & 0x0000FFFF; }
public int LowWord(this int number, int newValue)
{ return (number & 0xFFFF0000) + (newValue & 0x0000FFFF); }
public int HighWord(this int number)
{ return number & 0xFFFF0000; }
public int HighWord(this int number, int newValue)
{ return (number & 0x0000FFFF) + (newValue << 16); }
}
EDIT 2: On second thoughts, if you really need to do this and don't want the syntax everywhere, use Michael's solution. +1 to him for showing me something new.

I guess you don't want calculations when you want the Hiword / Hibyte or the LoWord / Lobyte,
if a System.Int32 starts at address 100 (so it occupies address 100 to 103), you want as a LoWord the two bytes starting at address 100 and 101 and Hiword is address 102 and 103.
This can be achieved using the class BitConverter. This class doesn't do anything with the bits, it only uses the addresses to return the requested value.
As the size of types like int / long are different per platform, and WORD and DWORD are a bit confusing, I use the System types System.Int16/Int32/Int64. No one will ever have any problems guessing the number of bits in a System.Int32.
With BitConverter you can convert any integer to the array of bytes starting on that location and convert an array of bytes of the proper length to the corresponding integer. No calculations needed and no bits will change,
Suppose you have a System.Int32 X (which is a DWORD in old terms)
LOWORD: System.Int16 y = BitConverter.ToInt16(BitConverter.GetBytes(x), 0);
HIWORD: System.Int16 y = BitConverter.ToInt16(BitConverter.GetBytes(x), 2);
The nice thing is that this works with all lengths, you don't have to combine functions like LOBYTE and HIWORD to get the third byte:
HIByte(Hiword(x)) will be like: BitConverter.GetBytes(x)[3]

Another Alternative
public class Macro
{
public static short MAKEWORD(byte a, byte b)
{
return ((short)(((byte)(a & 0xff)) | ((short)((byte)(b & 0xff))) << 8));
}
public static byte LOBYTE(short a)
{
return ((byte)(a & 0xff));
}
public static byte HIBYTE(short a)
{
return ((byte)(a >> 8));
}
public static int MAKELONG(short a, short b)
{
return (((int)(a & 0xffff)) | (((int)(b & 0xffff)) << 16));
}
public static short HIWORD(int a)
{
return ((short)(a >> 16));
}
public static short LOWORD(int a)
{
return ((short)(a & 0xffff));
}
}

I use these 2 function...
public static int GetHighint(long intValue)
{
return Convert.ToInt32(intValue >> 32);
}
public static int GetLowint(long intValue)
{
long tmp = intValue << 32;
return Convert.ToInt32(tmp >> 32);
}