I am using C# WPF to make a real-time FFT.
I am using NAudio's WaveIn and BufferedWaveProvider to capture any sound recorded by Stereo Mix. I take the FFT of the buffer many times per second and display it using a bitmap so that the display shows a real-time fourier transform of any audio playing through the speakers.
My problem is that, as expected, the displayed FFT lags behind the audio coming from the speakers by a small amount (maybe 200 ms).
Is there any way I can record the current audio that is supposed to be playing from the speakers so that I can perform the FFT on it and then play it back a small amount of time later (ex. 200 ms) while muting the original real-time audio.
The end result would simply be to effectively remove the perceived delay from the displayed FFT. Audio from a youtube video, for example, would lag slightly behind the video while my program is running.
Here are the relevant methods from what I have right now:
public MainWindow()
{
sampleSize = (int)Math.Pow(2, 13);
BUFFERSIZE = sampleSize * 2;
InitializeComponent();
// get the WaveIn class started
WaveIn wi = new WaveIn();
wi.DeviceNumber = deviceNo;
wi.WaveFormat = new NAudio.Wave.WaveFormat(RATE, WaveIn.GetCapabilities(wi.DeviceNumber).Channels);
// create a wave buffer and start the recording
wi.DataAvailable += new EventHandler<WaveInEventArgs>(wi_DataAvailable);
bwp = new BufferedWaveProvider(wi.WaveFormat);
bwp.BufferLength = BUFFERSIZE; //each sample is 2 bytes
bwp.DiscardOnBufferOverflow = true;
wi.StartRecording();
}
public void UpdateFFT()
{
// read the bytes from the stream
byte[] buffer = new byte[BUFFERSIZE];
bwp.Read(buffer, 0, BUFFERSIZE);
if (buffer[BUFFERSIZE - 2] == 0) return;
Int32[] vals = new Int32[sampleSize];
Ys = new double[sampleSize];
for (int i = 0; i < vals.Length; i++)
{
// bit shift the byte buffer into the right variable format
byte hByte = buffer[i * 2 + 1];
byte lByte = buffer[i * 2 + 0];
vals[i] = (short)((hByte << 8) | lByte);
Ys[i] = vals[i];
}
FFT(Ys);
}
I am still new to audio processing - any help would be appreciated.
The cause of your delay is the latency of WaveIn which is about 200ms by default. You can reduce that but at the risk of dropouts.
Whilst you can capture audio being played by the system with WasapiCapture there is no way to change it with NAudio, or delay its playback.
Related
This is similar to this question (kinda outdated), my problem is that I am trying to create an AudioClip from a float array that has been converted from byte array that has been converted from a base64 string. At the end it plays a loud, horrible and fast sound.
You can use this online tool to encode a small .wav file into a base64 string and this other online tool to make sure the large base64 string decoded generates the exact audio file.
I followed the other question solution and also tried changing the frequency value but it still plays a horrible sound. What Am I doing wrong?
AudioSource myAudio;
public float frequency = 44100; //Maybe 22050? since it is a wav mono sound
void Start(){
myAudio = GetComponent<AudioSource>();
}
//When clicking on the game object, play the sound
private void OnMouseDown(){
string audioAsString="UklGRjbPAQBXQVZFZm10IBIAAAABAAEAIlYAAESsAAACABAAAABkYXRh....."; //Base64 string encoded from the online tool. I can't put the whole string here because of max characters warning from StackOverflow questions.
byte[] byteArray = Convert.FromBase64String(audioAsString); //From Base64 string to byte[]
float[] floatArray = ConvertByteArrayToFloatArray(byteArray); //From byte[] to float[]
AudioClip audioClip = AudioClip.Create("test", floatArray.Length , 1, frequency, false);
audioClip.SetData(floatArray, 0);
myAudio.clip = audioClip;
myAudio.Play(); //Plays the audio
}
private float[] ConvertByteArrayToFloatArray(byte[] array)
{
float[] floatArr = new float[array.Length / 4];
for (int i = 0; i < floatArr.Length; i++)
{
if (BitConverter.IsLittleEndian) //I am still not sure what this line does
Array.Reverse(array, i * 4, 4);
floatArr[i] = BitConverter.ToSingle(array, i * 4) / 0x80000000;
}
return floatArr;
}
My goal is to be able to take a row of 70 pixels, analyze all 70 pixels for a certain kind of color, and then throw to another function if criteria are met. This needs to happen at least once every 50 milliseconds, and preferably even faster than that.
My current code looks like so
public void CaptureArea()
{
using (Bitmap capture = new Bitmap(70, 35))
{
using (Graphics g = Graphics.FromImage(capture))
{
for (int i = 0; i < 10; i++)
{
g.CopyFromScreen(copyPoint, pastePoint, new Size(70, 35));
evaluteBitmap(capture);
}
}
}
}
public void evaluteBitmap(Bitmap scanArea)
{
Rectangle rect = new Rectangle(0, 0, 70, 35);
BitmapData data = scanArea.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
IntPtr ptr = data.Scan0;
int bytes = data.Stride * scanArea.Height;
byte[] rgbValues = new byte[bytes];
byte[] r = new byte[bytes / 3];
byte[] g = new byte[bytes / 3];
byte[] b = new byte[bytes / 3];
Marshal.Copy(ptr, rgbValues, 0, bytes);
int count = 0;
int stride = data.Stride;
for (int column = 0; column < 30; column++)
{
b[count] = (byte)(rgbValues[(column * stride) + (34 * 3)]);
g[count] = (byte)(rgbValues[(column * stride) + (34 * 3) + 1]);
r[count++] = (byte)(rgbValues[(column * stride) + (34 * 3) + 2]);
}
scanArea.UnlockBits(data);
}
Just CaptureArea() can drop about 60 bitmaps into memory every second, which is fine, but it currently takes about 600 milliseconds for EvaluateBitmap() to grab a pixel and split it out into RGB values 70 times.
The end result is that a single frame of data takes well over 500 milliseconds to process when it needs to be much closer to 50 milliseconds. This kind of problem-solving is beyond me and I'm not experienced enough with code of this nature to be able to eyeball solutions and know what will end up being faster or slower and by how much.
Is there a way I can get performance quicker by an order of magnitude or am I asking the impossible?
Profiling session for the given code gives an unambiguous result:
CopyFromScreen - 40.00% exclusive samples
Bitmap..ctor - 15.00% exclusive samples
clr.dll - 11.67% exclusive samples
KernelBase.dll - 8.33% exclusive samples
GdiPlus.dll - 6.67% exclusive samples
LockBits - 6.67% exclusive samples
Image.Dispose - 5.00% exclusive samples
....
....
EvaluateBitmap - 1.67% exclusive samples
....
....
CaptureArea - 0.0% exclusive samples
The biggest portion of the time is spent on .NET methods which cannot be improved.
Conclusion: there no possible significant improvement of the code.
You can probably use multithreading to process more frames during the given time period, though.
I have some buffer containing samples of sinus which I want to be played from a memory stream periodically. Is there any efficient way to play it without having gaps in time? I try to make my own signal generator (I know that are some libraries providing that but I want to generate it by myself).
The platform is Windows Phone 8.1 silverlight
Update: the code is taken from this forum somewhere
public static void PlayBeep(UInt16 frequency, int msDuration, UInt16 volume = 16383)
{
var mStrm = new MemoryStream();
BinaryWriter writer = new BinaryWriter(mStrm);
const double TAU = 2 * Math.PI;
int samplesPerSecond = 44100;
{
double theta = frequency * TAU / (double)samplesPerSecond;
// 'volume' is UInt16 with range 0 thru Uint16.MaxValue ( = 65 535)
// we need 'amp' to have the range of 0 thru Int16.MaxValue ( = 32 767)
double amp = volume >> 2; // so we simply set amp = volume / 2
for (int step = 0; step < samples; step++)
{
short s = (short)(amp * Math.Sin(theta * (double)step));
writer.Write(s);
}
}
}
Here how I did it - just create a new SoundEffectInstance object and set it to the return value of SoundEffect.CreateInstance.
https://msdn.microsoft.com/en-us/library/dd940203.aspx
SoundEffect mySoundPlay = new SoundEffect(mStrm.ToArray(), 16000,AudioChannels.Mono);
SoundEffectInstance instance = mySoundPlay.CreateInstance();
instance.IsLooped = true;
instance.Play();
(Newbie question)
NAudio allows to start playing an MP3 file from a given position (by converting it from ms into bytes using Waveformat.AverageBytesPerSecond), but is it possible to make it stop playing exactly at another given position (in ms)? Do I have to somehow manipulate the wavestream or there are easier ways?
There is a solution using a Timer simultaneously together with starting playback and stopping playback after the timer ticks, but it doesn't produce reliable results at all.
I'd create a custom IWaveProvider that only returns a maximum of a specified number of bytes from Read. Then reposition your Mp3FileReader to the start, and pass it in to the custom trimming wave provider
Here's some completely untested example code to give you an idea.
class TrimWaveProvider
{
private readonly IWaveProvider source;
private int bytesRead;
private readonly int maxBytesToRead;
public TrimWaveProvider(IWaveProvider source, int maxBytesToRead)
{
this.source = source;
this.maxBytesToRead = maxBytesToRead;
}
public WaveFormat WaveFormat { get { return source.WaveFormat; } }
public int Read(byte[] buffer, int offset, int bytesToRead)
{
int bytesToReadThisTime = Math.Min(bytesToRead, maxBytesToRead - bytesRead);
int bytesReadThisTime = source.Read(buffer, offset, bytesToReadThisTime);
bytesRead += bytesReadThisTime;
return bytesReadThisTime;
}
}
// and call it like this...
var reader = new Mp3FileReader("myfile.mp3");
reader.Position = reader.WaveFormat.AverageBytesPerSecond * 3; // start 3 seconds in
// read 5 seconds
var trimmer = new TrimWaveProvider(reader, reader.WaveFormat.AverageBytesPerSecond * 5);
WaveOut waveOut = new WaveOut();
waveOut.Init(trimmer);
I want to change the bit rate of wave file.
so I searched in the net and I figure out that the wave file contain a header which is 44 bytes length , and the 25,26,27 and 28 byte are used to store the bit rate of wave file
so I take the wave and store it in an array of byte, then changes the value of bytes that used to store the bit rate of wave.
here is the code :
private int sampleRate;
private byte[] ByteArr;
private MemoryStream ByteMem;
ByteArr = null;
ByteMem = null;
ByteArr = File.ReadAllBytes(pathOfWav.Text);
sampleRate = BitConverter.ToInt32(ByteArr, 24) * 2;
Array.Copy(BitConverter.GetBytes(sampleRate), 0, ByteArr, 24, 4);
ByteMem = new MemoryStream(ByteArr);
here I stored the Wave file location on pathOfWav.Text which is a textBox, then I stored All the bytes of wave file in ByteArr then convert the 4 byte (from 25 to 28) to Int32 and multiply it by 2 to Increase the speed of speech and stored the value in sampleRate
after that I modify the previous ByteArr with the new value of Bit Rate sampleRate, then I instance a new MemoryStream .
my question is,, how to play the new Wave stream using Naudio ???
To change the bitrate of a WAV file you can't just update its format chunk. You actually have to re-encode it at a new sample-rate / bit-depth (assuming it is PCM), or with a different bitrate selected for your codec if it is not PCM. I have written an article here on converting between various audio formats, including converting between different flavours of PCM. The same article will also explain what to do if you meant changing the sample rate instead of bitrate.
Have you solved the issue ? As per your comment, if you need only to change the sampleRate, then why are you using NAudio ? You can use default available players such as MediaPlayer/SoundPlayer. If so, you can refer to the below code. I have added a method for changing the sample rate. Though you can write the waveFormat separately or append, I have only referred to sample rate and its dependent fields. I am reading the entire file, closing and then opening the same for writing part by part.
(Original Reference for 'WaveHeader Format' in C#: http://www.codeproject.com/Articles/15187/Concatenating-Wave-Files-Using-C-2005)
public void changeSampleRate(string waveFile, int sampleRate)
{
if (waveFile == null)
{
return;
}
/* you can add additional input validation code here */
/* open for reading */
FileStream fs = new FileStream(waveFile, FileMode.Open, FileAccess.Read);
/* get the channel and bits per sample value -> required for calculation */
BinaryReader br = new BinaryReader(fs);
int length = (int)fs.Length - 8;
fs.Position = 22;
short channels = br.ReadInt16();
fs.Position = 34;
short BitsPerSample = br.ReadInt16();
byte[] arrfile = new byte[fs.Length];
fs.Position = 0;
fs.Read(arrfile, 0, arrfile.Length); /* read entire file */
br.Close();
fs.Close();
/* now open for writing */
fs = new FileStream(waveFile, FileMode.Open, FileAccess.Write);
BinaryWriter bw = new BinaryWriter(fs);
bw.BaseStream.Seek(0, SeekOrigin.Begin);
bw.Write(arrfile, 0, 24); //no change till this point
/* refer to waveFormat header */
bw.Write(sampleRate);
bw.Write((int)(sampleRate * ((BitsPerSample * channels) / 8)));
bw.Write((short)((BitsPerSample * channels) / 8));
/* you can keep the same data from here */
bw.Write(arrfile, 34, arrfile.Length - 34);
bw.Close();
fs.Close();
}
Now you can call the above method and play the wave file with different sample rates:
changeSampleRate(yourWaveFileToPlay, requiredSampleRate);
MediaPlayer mp = new MediaPlayer();
mp.Open(new Uri(yourWaveFileToPlay, UriKind.Absolute));
mp.Play();