We Keep Coding

Get PCM byte array from MediaFoundationResampler, Naudio

I'm working on a method to resample a wav file, here's the method:
internal byte[] ResampleWav(byte[] rawPcmData, int frequency, int bits, int channels, int newFrequency)
{
byte[] pcmData;
using (MemoryStream AudioSample = new MemoryStream(rawPcmData))
{
RawSourceWaveStream Original = new RawSourceWaveStream(AudioSample, new WaveFormat(frequency, bits, channels));
using (MediaFoundationResampler conversionStream = new MediaFoundationResampler(Original, new WaveFormat(newFrequency, bits, channels)))
{
// Here should go the code to get the array of bytes with the resampled PCM data
}
}
return pcmData;
}
The problem here is that there isn't any property in the MediaFoundationResampler that returns the size of the buffer. The method should return an array of bytes with the resampled PCM data only.
Thanks in advance!
--Edit
After some time working, I could get this:
internal byte[] WavChangeFrequency(byte[] rawPcmData, int frequency, int bits, int channels, int newFrequency)
{
byte[] pcmData;
using (MemoryStream AudioSample = new MemoryStream(rawPcmData))
{
RawSourceWaveStream Original = new RawSourceWaveStream(AudioSample, new WaveFormat(frequency, bits, channels));
using (MediaFoundationResampler conversionStream = new MediaFoundationResampler(Original, newFrequency))
{
//Start reading PCM data
using (MemoryStream wavData = new MemoryStream())
{
byte[] readBuffer = new byte[1024];
while ((conversionStream.Read(readBuffer, 0, readBuffer.Length)) != 0)
{
wavData.Write(readBuffer, 0, readBuffer.Length);
}
pcmData = wavData.ToArray();
}
}
}
return pcmData;
}
"Seems" to work fine, but there's another problem, seems that the PCM data byte array is greater than expected. Here's one of the tests I've tested with the method:
Input settings:
44100Hz
16 Bits
01 Channel
1846324 Bytes of PCM data
Expected (when I resample the same wav file with Audition, Audacity and WaveFormatConversionStream I get this):
22050Hz
16 Bits
01 Channel
923162 Bytes
MediaFoundationResampler result:
22050Hz
16 Bits
01 Channel
923648 Bytes
And the size changes drastically if I change the size of the readBuffer array.
The main problem is that MediaFoundationResampler doesn't have the property Length to know the real size of the resampled PCM data buffer. Using WaveFormatConversionStream the code would be this, but the quality is not very good:
internal byte[] WavChangeFrequency(byte[] rawPcmData, int frequency, int bits, int channels, int newFrequency)
{
byte[] pcmData;
using (MemoryStream AudioSample = new MemoryStream(rawPcmData))
{
RawSourceWaveStream Original = new RawSourceWaveStream(AudioSample, new WaveFormat(frequency, bits, channels));
using (WaveFormatConversionStream wavResampler = new WaveFormatConversionStream(new WaveFormat(newFrequency, bits, channels), Original))
{
pcmData = new byte[wavResampler.Length];
wavResampler.Read(pcmData, 0, pcmData.Length);
}
}
return pcmData;
}
What should I do to get the expected PCM data array, using the MediaFoundationResampler?

Disclaimer
I'm not familiar with the NAudio Library, so there might be a more proper way of doing this.
EDIT
Still not a good answer, seems still off by a few bytes...
Some corrections to the code, using Mark Heath (NAudio creator) comment on this answer: https://stackoverflow.com/a/14481756/9658671
I keep the answer here for now, as it might help for finding a real answer, but I'll edit or remove it if necessary.
/EDIT
The difference in length between the file produced by Audition and your code is 923648 - 923162 = 486 bytes, which is less than your 1024 buffer.
It can be explained by the following mechanism:
At the very last call to the Read method, the remaining byte count is inferior to your buffer size. So instead of getting 1024 bytes, you get less.
But your code still adds a full 1024 byte group, instead of a smaller number. That explains the 486 bytes difference and the fact that this number will change if you choose another buffer size.
Fixing this should be easy.
From NAudio documentation:
https://github.com/naudio/NAudio/blob/fb35ce8367f30b8bc5ea84e7d2529e172cf4c381/Docs/WaveProviders.md
The Read method returns the number for bytes that were read. This
should never be more than numBytes and can only be less if the end of
the audio stream is reached. NAudio playback devices will stop playing
when Read returns 0.
So instead of pushing always 1024 bytes at each iteration, just push the number returned by the Read method.
Also, from Mark Heath comment:
the buffer size should be configurable to be an exact multiple of the
block align of the WaveStream
So instead of choosing a "random" buffer size, use a multiple of the block align.
internal byte[] WavChangeFrequency(byte[] rawPcmData, int frequency, int bits, int channels, int newFrequency, int BlockAlign)
{
byte[] pcmData;
var BufferSize = BlockAlign * 1024;
using (MemoryStream AudioSample = new MemoryStream(rawPcmData))
{
RawSourceWaveStream Original = new RawSourceWaveStream(AudioSample, new WaveFormat(frequency, bits, channels));
using (MediaFoundationResampler conversionStream = new MediaFoundationResampler(Original, newFrequency))
{
//Start reading PCM data
using (MemoryStream wavData = new MemoryStream())
{
var ByteCount = 0;
var readBuffer = new byte[BufferSize];
while ((ByteCount = conversionStream.Read(readBuffer, 0, readBuffer.Length)) != 0)
{
wavData.Write(readBuffer, 0, ByteCount);
}
pcmData = wavData.ToArray();
}
}
}
return pcmData;
}

BluetoothLEAdvertisementDataSection (ArgumentException)

I'm trying to advertise an Eddystone beacon but my code fails at advertisementData.Data with an ArgumentException:
Value does not fall within the expected range.
Any ideas of what's happening?
// ...
using (var memoryStream = new MemoryStream())
{
byte messageLengthByte = Convert.ToByte(message.Length);
memoryStream.WriteByte(messageLengthByte);
memoryStream.Write(message, 0, message.Length);
fullMessage = memoryStream.ToArray();
}
while (fullMessage.Length < 32)
{
byte[] newArray = new byte[fullMessage.Length + 1];
fullMessage.CopyTo(newArray, 0);
newArray[fullMessage.Length] = 0x00;
fullMessage = newArray;
}
var writer = new DataWriter();
writer.WriteBytes(fullMessage);
var advertisementData = new BluetoothLEAdvertisementDataSection();
advertisementData.Data = writer.DetachBuffer(); // Error!
publisher.Advertisement.DataSections.Add(advertisementData);
publisher.Start();

Most likely you're trying to fit in more bytes than the BLE packet allows. The max size is 32 bytes, but that's including:
3 bytes for the "flags" data section, which I believe is mandatory and might be automatically set by the Windows 10 BLE API
for each additional section, 1 byte for the length of the section, and 1 bytes for the type of the section
If you only broadcast a single section, that leaves you with 27 bytes for that section's actual payload.

Copy all but the last 16 bytes of a stream? Early detection of end-of-stream?

This is C# related. We have a case where we need to copy the entire source stream into a destination stream except for the last 16 bytes.
EDIT: The streams can range upto 40GB, so can't do some static byte[] allocation (eg: .ToArray())
Looking at the MSDN documentation, it seems that we can reliably determine the end of stream only when the return value is 0. Return values between 0 and the requested size can imply bytes are "not currently available" (what does that really mean?)
Currently it copies every single byte as follows. inStream and outStream are generic - can be memory, disk or network streams (actually some more too).
public static void StreamCopy(Stream inStream, Stream outStream)
{
var buffer = new byte[8*1024];
var last16Bytes = new byte[16];
int bytesRead;
while ((bytesRead = inStream.Read(buffer, 0, buffer.Length)) > 0)
{
outStream.Write(buffer, 0, bytesRead);
}
// Issues:
// 1. We already wrote the last 16 bytes into
// outStream (possibly over the n/w)
// 2. last16Bytes = ? (inStream may not necessarily support rewinding)
}
What is a reliable way to ensure all but the last 16 are copied? I can think of using Position and Length on the inStream but there is a gotcha on MSDN that says
If a class derived from Stream does not support seeking, calls to Length, SetLength, Position, and Seek throw a NotSupportedException. .

Read between 1 and n bytes from the input stream.1
Append the bytes to a circular buffer.2
Write the first max(0, b - 16) bytes from the circular buffer to the output stream, where b is the number of bytes in the circular buffer.
Remove the bytes that you just have written from the circular buffer.
Go to step 1.
1This is what the Read method does – if you call int n = Read(buffer, 0, 500); it will read between 1 and 500 bytes into buffer and return the number of bytes read. If Read returns 0, you have reached the end of the stream.
2For maximum performance, you can read the bytes directly from the input stream into the circular buffer. This is a bit tricky, because you have to deal with the wraparound within the array underlying the buffer.

The following solution is fast and tested. Hope it's useful. It uses the double buffering idea you already had in mind. EDIT: simplified loop removing the conditional that separated the first iteration from the rest.
public static void StreamCopy(Stream inStream, Stream outStream) {
// Define the size of the chunk to copy during each iteration (1 KiB)
const int blockSize = 1024;
const int bytesToOmit = 16;
const int buffSize = blockSize + bytesToOmit;
// Generate working buffers
byte[] buffer1 = new byte[buffSize];
byte[] buffer2 = new byte[buffSize];
// Initialize first iteration
byte[] curBuffer = buffer1;
byte[] prevBuffer = null;
int bytesRead;
// Attempt to fully fill the buffer
bytesRead = inStream.Read(curBuffer, 0, buffSize);
if( bytesRead == buffSize ) {
// We succesfully retrieved a whole buffer, we will output
// only [blockSize] bytes, to avoid writing to the last
// bytes in the buffer in case the remaining 16 bytes happen to
// be the last ones
outStream.Write(curBuffer, 0, blockSize);
} else {
// We couldn't retrieve the whole buffer
int bytesToWrite = bytesRead - bytesToOmit;
if( bytesToWrite > 0 ) {
outStream.Write(curBuffer, 0, bytesToWrite);
}
// There's no more data to process
return;
}
curBuffer = buffer2;
prevBuffer = buffer1;
while( true ) {
// Attempt again to fully fill the buffer
bytesRead = inStream.Read(curBuffer, 0, buffSize);
if( bytesRead == buffSize ) {
// We retrieved the whole buffer, output first the last 16
// bytes of the previous buffer, and output just [blockSize]
// bytes from the current buffer
outStream.Write(prevBuffer, blockSize, bytesToOmit);
outStream.Write(curBuffer, 0, blockSize);
} else {
// We could not retrieve a complete buffer
if( bytesRead <= bytesToOmit ) {
// The bytes to output come solely from the previous buffer
outStream.Write(prevBuffer, blockSize, bytesRead);
} else {
// The bytes to output come from the previous buffer and
// the current buffer
outStream.Write(prevBuffer, blockSize, bytesToOmit);
outStream.Write(curBuffer, 0, bytesRead - bytesToOmit);
}
break;
}
// swap buffers for next iteration
byte[] swap = prevBuffer;
prevBuffer = curBuffer;
curBuffer = swap;
}
}
static void Assert(Stream inStream, Stream outStream) {
// Routine that tests the copy worked as expected
inStream.Seek(0, SeekOrigin.Begin);
outStream.Seek(0, SeekOrigin.Begin);
Debug.Assert(outStream.Length == Math.Max(inStream.Length - bytesToOmit, 0));
for( int i = 0; i < outStream.Length; i++ ) {
int byte1 = inStream.ReadByte();
int byte2 = outStream.ReadByte();
Debug.Assert(byte1 == byte2);
}
}
A much easier solution to code, yet slower since it would work at a byte level, would be to use an intermediate queue between the input stream and the output stream. The process would first read and enqueue 16 bytes from the input stream. Then it would iterate over the remaining input bytes, reading a single byte from the input stream, enqueuing it and then dequeuing a byte. The dequeued byte would be written to the output stream, until all bytes from the input stream are processed. The unwanted 16 bytes should linger in the intermediate queue.
Hope this helps!
=)

Use a circular buffer sounds great but there is no circular buffer class in .NET which means additional code anyways. I ended up with the following algorithm, a sort of map and copy - I think it's simple. The variable names are longer than usual for the sake of being self descriptive here.
This flows thru the buffers as
[outStream] <== [tailBuf] <== [mainBuf] <== [inStream]
public byte[] CopyStreamExtractLastBytes(Stream inStream, Stream outStream,
int extractByteCount)
{
//var mainBuf = new byte[1024*4]; // 4K buffer ok for network too
var mainBuf = new byte[4651]; // nearby prime for testing
int mainBufValidCount;
var tailBuf = new byte[extractByteCount];
int tailBufValidCount = 0;
while ((mainBufValidCount = inStream.Read(mainBuf, 0, mainBuf.Length)) > 0)
{
// Map: how much of what (passthru/tail) lives where (MainBuf/tailBuf)
// more than tail is passthru
int totalPassthruCount = Math.Max(0, tailBufValidCount +
mainBufValidCount - extractByteCount);
int tailBufPassthruCount = Math.Min(tailBufValidCount, totalPassthruCount);
int tailBufTailCount = tailBufValidCount - tailBufPassthruCount;
int mainBufPassthruCount = totalPassthruCount - tailBufPassthruCount;
int mainBufResidualCount = mainBufValidCount - mainBufPassthruCount;
// Copy: Passthru must be flushed per FIFO order (tailBuf then mainBuf)
outStream.Write(tailBuf, 0, tailBufPassthruCount);
outStream.Write(mainBuf, 0, mainBufPassthruCount);
// Copy: Now reassemble/compact tail into tailBuf
var tempResidualBuf = new byte[extractByteCount];
Array.Copy(tailBuf, tailBufPassthruCount, tempResidualBuf, 0,
tailBufTailCount);
Array.Copy(mainBuf, mainBufPassthruCount, tempResidualBuf,
tailBufTailCount, mainBufResidualCount);
tailBufValidCount = tailBufTailCount + mainBufResidualCount;
tailBuf = tempResidualBuf;
}
return tailBuf;
}

Convert memorystream to double-array?

I've got a pcm-stream of raw music data from a wave-file and would like to convert it to a double-array (to apply a fft afterwards).
The result I got now contains very high or low double-numbers (1.0E-200 and 1.0E+300) and I'm not sure whether these can be correct.
This is the code I'm using right now:
WaveStream pcm = WaveFormatConversionStream.CreatePcmStream(mp3);
double[] real = new double[pcm.Length];
byte[] buffer = new byte[8];
int count = 0;
while ((read = pcm.Read(buffer, 0, buffer.Length)) > 0)
{
real[count] = BitConverter.ToDouble(buffer, 0);
count++;
}

Your PCM stream is almost certainly 16 bit. So instead of BitConverter.ToDouble use ToInt16 instead. Then divide by 32768.0 to get into the the range +/- 1.0

I realize that this question is old; however, I thought I might offer this alternative approach to calling BitConverter.ToDouble.
public static double[] ToDoubleArray(this byte[] bytes)
{
Debug.Assert(bytes.Length % sizeof(double) == 0, "byte array must be aligned on the size of a double.");
double[] doubles = new double[bytes.Length / sizeof(double)];
GCHandle pinnedDoubles = GCHandle.Alloc(doubles, GCHandleType.Pinned);
Marshal.Copy(bytes, 0, pinnedDoubles.AddrOfPinnedObject(), bytes.Length);
pinnedDoubles.Free();
return doubles;
}
public static double[] ToDoubleArray(this MemoryStream stream)
{
return stream.ToArray().ToDoubleArray();
}

An elegant way to consume (all bytes of a) BinaryReader?

Is there an elegant to emulate the StreamReader.ReadToEnd method with BinaryReader? Perhaps to put all the bytes into a byte array?
I do this:
read1.ReadBytes((int)read1.BaseStream.Length);
...but there must be a better way.

Original Answer (Read Update Below!)
Simply do:
byte[] allData = read1.ReadBytes(int.MaxValue);
The documentation says that it will read all bytes until the end of the stream is reached.
Update
Although this seems elegant, and the documentation seems to indicate that this would work, the actual implementation (checked in .NET 2, 3.5, and 4) allocates a full-size byte array for the data, which will probably cause an OutOfMemoryException on a 32-bit system.
Therefore, I would say that actually there isn't an elegant way.
Instead, I would recommend the following variation of #iano's answer. This variant doesn't rely on .NET 4:
Create an extension method for BinaryReader (or Stream, the code is the same for either).
public static byte[] ReadAllBytes(this BinaryReader reader)
{
const int bufferSize = 4096;
using (var ms = new MemoryStream())
{
byte[] buffer = new byte[bufferSize];
int count;
while ((count = reader.Read(buffer, 0, buffer.Length)) != 0)
ms.Write(buffer, 0, count);
return ms.ToArray();
}
}

There is not an easy way to do this with BinaryReader. If you don't know the count you need to read ahead of time, a better bet is to use MemoryStream:
public byte[] ReadAllBytes(Stream stream)
{
using (var ms = new MemoryStream())
{
stream.CopyTo(ms);
return ms.ToArray();
}
}
To avoid the additional copy when calling ToArray(), you could instead return the Position and buffer, via GetBuffer().

To copy the content of a stream to another, I've solved reading "some" bytes until the end of the file is reached:
private const int READ_BUFFER_SIZE = 1024;
using (BinaryReader reader = new BinaryReader(responseStream))
{
using (BinaryWriter writer = new BinaryWriter(File.Open(localPath, FileMode.Create)))
{
int byteRead = 0;
do
{
byte[] buffer = reader.ReadBytes(READ_BUFFER_SIZE);
byteRead = buffer.Length;
writer.Write(buffer);
byteTransfered += byteRead;
} while (byteRead == READ_BUFFER_SIZE);
}
}

Had the same problem.
First, get the file's size using FileInfo.Length.
Next, create a byte array and set its value to BinaryReader.ReadBytes(FileInfo.Length).
e.g.
var size = new FileInfo(yourImagePath).Length;
byte[] allBytes = yourReader.ReadBytes(System.Convert.ToInt32(size));

Another approach to this problem is to use C# extension methods:
public static class StreamHelpers
{
public static byte[] ReadAllBytes(this BinaryReader reader)
{
// Pre .Net version 4.0
const int bufferSize = 4096;
using (var ms = new MemoryStream())
{
byte[] buffer = new byte[bufferSize];
int count;
while ((count = reader.Read(buffer, 0, buffer.Length)) != 0)
ms.Write(buffer, 0, count);
return ms.ToArray();
}
// .Net 4.0 or Newer
using (var ms = new MemoryStream())
{
stream.CopyTo(ms);
return ms.ToArray();
}
}
}
Using this approach will allow for both reusable as well as readable code.

I use this, which utilizes the underlying BaseStream property to give you the length info you need. It keeps things nice and simple.
Below are three extension methods on BinaryReader:
The first reads from wherever the stream's current position is to the end
The second reads the entire stream in one go
The third utilizes the Range type to specify the subset of data you are interested in.
public static class BinaryReaderExtensions {
public static byte[] ReadBytesToEnd(this BinaryReader binaryReader) {
var length = binaryReader.BaseStream.Length - binaryReader.BaseStream.Position;
return binaryReader.ReadBytes((int)length);
}
public static byte[] ReadAllBytes(this BinaryReader binaryReader) {
binaryReader.BaseStream.Position = 0;
return binaryReader.ReadBytes((int)binaryReader.BaseStream.Length);
}
public static byte[] ReadBytes(this BinaryReader binaryReader, Range range) {
var (offset, length) = range.GetOffsetAndLength((int)binaryReader.BaseStream.Length);
binaryReader.BaseStream.Position = offset;
return binaryReader.ReadBytes(length);
}
}
Using them is then trivial and clear...
// 1 - Reads everything in as a byte array
var rawBytes = myBinaryReader.ReadAllBytes();
// 2 - Reads a string, then reads the remaining data as a byte array
var someString = myBinaryReader.ReadString();
var rawBytes = myBinaryReader.ReadBytesToEnd();
// 3 - Uses a range to read the last 44 bytes
var rawBytes = myBinaryReader.ReadBytes(^44..);