We Keep Coding

Reading a MemoryStream containing opus audio with NAudio

I'm trying to play opus audio files from web which I try to buffer with a MemoryStream. I'm aware of NAudio's ability to take urls however I need to set cookies and user agent before I access the file so this eliminates that option.
My latest approach was to buffer 30~ seconds of stream, feed it to StreamMediaFoundationReader and write to the same MemoryStream when needed, however NAudio ends up playing the initial buffered segment and stop after that is completed. What would be the correct approach for this?
Here's my current code if needed. (I have no idea how audio streaming works so please go easy on me)
bufstr.setReadStream(httpreq.GetResponse().GetResponseStream()); //bufstr is a custom class which creates a memorystream I can write to.
StreamMediaFoundationReader streamread = new StreamMediaFoundationReader(bufstr.getStream());
bufstr.readToStream(); //get the initial 30~ seconds of content
waveOut.Init(streamread);
waveOut.Play();
int seconds = 0;
while(waveOut.PlaybackState == PlaybackState.Playing) {
Thread.Sleep(1000);
seconds++;
if (secs % 30 > 15) bufstr.readToStream();
}
bufstr's readToStream method
public void readToStream() {
int prevbufcount = totalbuffered; //I keep track of how many bytes I fetched from remote url.
while (stream.CanRead && prevbufcount + (30 * (this.bitrate / 8)) > totalbuffered && totalbuffered != contentlength) { //read around 30 seconds of content;
Console.Write($"Caching {prevbufcount + (30 * (this.bitrate / 8))}/{totalbuffered}\r");
byte[] buf = new byte[1024];
bufferedcount = stream.Read(buf, 0, 1024);
totalbuffered += bufferedcount;
memorystream.Write(buf, 0, bufferedcount);
}
}

While debugging I found out that content length I get from server does not match with the actual size of stream, so I ended up calculating content length via other details I get from server.
The issue might also be a race condition due to the fact that it stopped after I manually kept track of where I write on memory stream

How to decompress just one frame using NAudio?

I am simply trying to take any or first frame from an mp3 file and then decompress it.
internal void Read3(string location) //take in file location
{
Mp3FileReader mp3 = new Mp3FileReader(location); //make a Mp3FileReader
SECTIONA: //to jump back here when needed.
byte[] _passedBuffer = Decompress(mp3.ReadNextFrame()); //passed the decompressed byte array here.
int jump_size = mp3.WaveFormat.Channels *2; //just to get how many bytes to skip
for (int i = 0; i < _passedBuffer.Length; i += jump_size)
{
short FinalSample = BitConverter.ToInt16(_passedBuffer, i);
if (jump_size == 4) //converting the bytes to Int16,nothing special here.
{
FinalSample = (short)(((BitConverter.ToInt16(_passedBuffer, i + 2)) + FinalSample) / 2);
}
Console.Write(FinalSample+"|"); //and writing it down to Console.
}
Console.WriteLine("Frames are Written,continue to next frame?");
if (Convert.ToChar(Console.Read()) == 'y') //asking to go on or not.
{ goto SECTIONA; }
}
private byte[] Decompress(Mp3Frame fm)
{
var buffer = new byte[16384 * 4]; //big enough buffer size
WaveFormat wf = new Mp3WaveFormat(fm.SampleRate, fm.ChannelMode == ChannelMode.Mono ? 1 : 2, fm.FrameLength, fm.BitRate); //creating a new WaveFormat
IMp3FrameDecompressor decompressor = new AcmMp3FrameDecompressor(wf); //passing in to AcmMp3FrameDecompressor.
decompressor.DecompressFrame(fm, buffer, 0); //running the DecompressFrame method and then passing back the buffer.
return buffer;
}
Now the Mp3FileReader is reading the Frame correctly as I checked the Frame's RawData. Now I am trying to decompress that Frame and then convert its PCM data into Int16 in that only For Loop but every Int16 FinalSample value is returning 0.
I know that just using Mp3FileReader.Read(Buffer,Offset,Length) will get the job done but for all the frames so:
how do I do it for just one frame?
what is wrong with my code because of which I am getting all zeros?
I know that RawData is ok, so something must be wrong with Decompress method, How do I setup a decompressor for mp3 file?

You can use the AcmMp3FrameDecompressor or DmoMp3FrameDecompressor to decompress individual MP3 frames.
You need to check the return value of Decompress to see how much data was returned. It's possible for the first frame not to return anything.
You should also create a single frame decompressor and use it for all frames in the MP3 file, as it retains state between calls.

Audio Encoding conversion problems with PCM 32-bit yo PCM 16-bit

I am using C# in Universal Windows App to write a Watson Speech-to-text service.
For now instead of using the Watson service, I write to the file and then read it in the Audacity to confirm it is in the right format since Watson service wasn't returning correct responses to me, and the following explains why.
For some reason when I create 16-bit PCM encoding properties, and read buffer, I am only able to read data as 32-bit PCM, and it's working well, but if I read it in 16-bit PCM it is in slow motion, and all the speech is basically corrupted.
I don't really know what exactly needs to be done to convert from 32-bit to 16-bit, but here's what I have in my C# application:
//Creating PCM Encoding properties
var pcmEncoding = AudioEncodingProperties.CreatePcm(16000, 1, 16);
var result = await AudioGraph.CreateAsync(
new AudioGraphSettings(AudioRenderCategory.Speech)
{
DesiredRenderDeviceAudioProcessing = AudioProcessing.Raw,
AudioRenderCategory = AudioRenderCategory.Speech,
EncodingProperties = pcmEncoding
}
);
graph = result.Graph;
//Initialize microphone
var microphone = await DeviceInformation.CreateFromIdAsync(MediaDevice.GetDefaultAudioCaptureId(AudioDeviceRole.Default));
var micInputResult = await graph.CreateDeviceInputNodeAsync(MediaCategory.Speech, pcmEncoding, microphone);
//Create frame output node
frameOutputNode = graph.CreateFrameOutputNode(pcmEncoding);
//Callback function to fire when buffer is filled with data
graph.QuantumProcessed += (s, a) => ProcessFrameOutput(frameOutputNode.GetFrame());
frameOutputNode.Start();
//Make the microphone write into the frame node
micInputResult.DeviceInputNode.AddOutgoingConnection(frameOutputNode);
micInputResult.DeviceInputNode.Start();
graph.Start();
Initialization step is done at this stage. Now, actually reading from the buffer and writing to the file is only working if I use 32-bit PCM encoding with the following function (commented out is the PCM 16-bit code that is resulting in a slow motion speech output):
private void ProcessFrameOutput(AudioFrame frame)
{
//Making a copy of the audio frame buffer
var audioBuffer = frame.LockBuffer(AudioBufferAccessMode.Read);
var buffer = Windows.Storage.Streams.Buffer.CreateCopyFromMemoryBuffer(audioBuffer);
buffer.Length = audioBuffer.Length;
using (var dataReader = DataReader.FromBuffer(buffer))
{
dataReader.ByteOrder = ByteOrder.LittleEndian;
byte[] byteData = new byte[buffer.Length];
int pos = 0;
while (dataReader.UnconsumedBufferLength > 0)
{
/*Reading Float -> Int 32*/
/*With this code I can import raw wav file into the Audacity
using Signed 32-bit PCM Encoding, and it is working well*/
var singleTmp = dataReader.ReadSingle();
var int32Tmp = (Int32)(singleTmp * Int32.MaxValue);
byte[] chunkBytes = BitConverter.GetBytes(int32Tmp);
byteData[pos++] = chunkBytes[0];
byteData[pos++] = chunkBytes[1];
byteData[pos++] = chunkBytes[2];
byteData[pos++] = chunkBytes[3];
/*Reading Float -> Int 16 (Slow Motion)*/
/*With this code I can import raw wav file into the Audacity
using Signed 16-bit PCM Encoding, but when I play it, it's in
a slow motion*/
//var singleTmp = dataReader.ReadSingle();
//var int16Tmp = (Int16)(singleTmp * Int16.MaxValue);
//byte[] chunkBytes = BitConverter.GetBytes(int16Tmp);
//byteData[pos++] = chunkBytes[0];
//byteData[pos++] = chunkBytes[1];
}
WriteBytesToFile(byteData);
}
}
Can anyone think of a reason why this is happening? Is it because Int32 PCM is larger in size and when I use Int16, it extends it and makes the sound longer? Or am I not sampling it properly?
Note: I tried reading Bytes directly from the buffer, and then using that as a raw data, but it's not encoded as PCM that way.
Reading Int16/32 from the buffer directly also doesn't work.
In the above example I am only using Frame Output node. IF I create a file output node that automatically writes to the raw file, it works really well as 16-bit PCM, so something is wrong in my callback function that causes it to be in a slow motion.
Thanks

//Creating PCM Encoding properties
var pcmEncoding = AudioEncodingProperties.CreatePcm(16000, 1, 16);
var result = await AudioGraph.CreateAsync(
new AudioGraphSettings(AudioRenderCategory.Speech)
{
DesiredRenderDeviceAudioProcessing = AudioProcessing.Raw,
AudioRenderCategory = AudioRenderCategory.Speech,
EncodingProperties = pcmEncoding
}
);
graph = result.Graph;
pcmEncoding does not make much sense here since only Float encoding is supported by AudioGraph.
byte[] byteData = new byte[buffer.Length];
it should be buffer.Length / 2 since you convert from float data with 4 bytes per sample to int16 data with 2 bytes per sample
/*Reading Float -> Int 16 (Slow Motion)*/
/*With this code I can import raw wav file into the Audacity
using Signed 16-bit PCM Encoding, but when I play it, it's in
a slow motion*/
var singleTmp = dataReader.ReadSingle();
var int16Tmp = (Int16)(singleTmp * Int16.MaxValue);
byte[] chunkBytes = BitConverter.GetBytes(int16Tmp);
byteData[pos++] = chunkBytes[0];
byteData[pos++] = chunkBytes[1];
This is correct code, it should work. Your "slow motion" is most likely related to the buffer size you incorrectly set before.
I must admit Microsoft needs someone to review their bloated APIs

Resampling raw audio with NAudio

I'd like to resample audio: change its sample rate from 44k to 11k. The input I've got is raw audio in bytes. It really is raw, it has no headers - if I try loading it into a WaveFileReader, I get an exception saying "Not a WAVE file - no RIFF header".
How I'm currently trying to achieve it is something like this (just a really simplified piece of code):
WaveFormat ResampleInputFormat = new WaveFormat(44100, 1);
WaveFormat ResampleOutputFormat = new WaveFormat(11025, 1);
MemoryStream ResampleInputMemoryStream = new MemoryStream();
foreach (var b in InputListOfBytes)
{
ResampleInputMemoryStream.Write(new byte[]{b}, 0, 1);
}
RawSourceWaveStream ResampleInputWaveStream =
new RawSourceWaveStream(ResampleInputMemoryStream, ResampleInputFormat);
WaveFormatConversionStream ResampleOutputStream =
new WaveFormatConversionStream(ResampleOutputFormat, ResampleInputWaveStream);
byte[] bytes = new byte[2];
while (ResampleOutputStream.Read(bytes, 0, 2) > 0)
{
OutputListOfBytes.Add(bytes[0]);
OutputListOfBytes.Add(bytes[1]);
}
My problem is: the last loop is an infinite loop. The Read() always gets the same values and never advances in the Stream. I even tried Seek()-ing to the right position after each Read(), but that doesn't seem to work either, I still always get the same values.
What am I doing wrong? And is this even the right way to resample raw audio? Thanks in advance!

First, you need to reset ResampleInputMemoryStream's position to the start. It may actually have been easier to create the memory stream based on the array:
new MemoryStream(InputListOfBytes)
Second, when reading out of the resampler, you need to read in larger chunks than two bytes at a time. Try at least a second's worth of audio (use ResampleOutputStream.WaveFormat.AverageBytesPerSecond).

bytes to image converting error

am crteating a client server application
and the client will ask the server for a certain image, and the server will send it to the client
when the client receive it , it will show it in a picturebox
so this is my code
string line = null;
line = textBox3.Text;
socket.Send(Encoding.ASCII.GetBytes(line));
data = new byte[1024];
dataSize = socket.Receive(data);
//string s = Encoding.ASCII.GetString(data, 0, dataSize);
// textBox4.Text = s;
Image newImage;
using (MemoryStream ms = new MemoryStream(data,0,dataSize))
{
ms.Write(data,0,dataSize);
newImage = Image.FromStream(ms,true); //HERE I GOT THE PROBLEM
}
pictureBox1.Image = newImage;
}
then it returns an error called, Parameter is not valid, so i dont know what wrong in here?

Hard to believe the image is less than 1KB in size. Have bigger buffer:
data = new byte[1024 * 500]; //limit to 500KB
Having buffer smaller than the actual size of the image probably results in an incomplete data which is indeed invalid stream for the image.

You need to reset the memory stream's position back to the start after writing to it:
...
ms.Write(data,0,dataSize);
ms.Position = 0;
newImage = Image.FromStream(ms,true); //HERE I GOT THE PROBLEM
...

Your network code is buggy in two ways:
1) If the data is >1024 bytes it won't work at all
2) If the incoming data gets fragmented it's break (One Send call does NOT map to one Receive call). TCP is a stream not packet based protocol.
To fix it first write the bytesize of the image, and when reading read until you have enough bytes and only then construct the image from the bytes.