Hello
I'm exploring the audio possibilities of the WP7 platform and the first stumble I've had is trying to implement a FFT using the Cooley-Tukey method. The result of that is that the spectrogram shows 4 identical images in this order: one normal, one reversed, one normal, one reversed.
The code was taken from another C# project (for desktop), the implementation and all variables seem in place with the algorithm.
So I can see two problems right away: reduced resolution and CPU wasted to generate four identical spectrograms.
Given a sample size of 1600 (could be 2048) I know have only 512 usable frequency information which leaves me with a 15Hz resolution for an 8kHz frequency span. Not bad, but not so good either.
Should I just give up on the code and use NAudio? I cannot seem to have an explanation why the spectrum is quadrupled, input data is ok, algorithm seems ok.
This sounds correct. You have 2 mirrors, I can only assume that one is the Real part and the other is the Image part. This is standard FFT.
From the real and image you can compute the magnitude or amplitude of each harmonic which is more common or compute the angle or phase shift of each harmonic which is less common.
Gilad.
I switched to NAudio and now the FFT works. However I might have found the cause (I probably won't try to test again): when I was constructing an array of double to feed into the FFT function, I did something like:
for (int i = 0; i < buffer.Length; i+= sizeof(short))
{
samples[i] = ReadSample(buffer, i);
}
For reference, 'samples' is the double[] input to fft, ReadSample is something that takes care of little/big endian. Can't remember right now how the code was, but it was skipping every odd sample.
My math knowledge has never been great but I'm thinking this induces some aliasing patterns which might in the end produce the effect I experienced.
Anyway, problem worked around, but thanks for your input and if you can still explain the phenomenon I am grateful.
Related
The float array buffers I'm getting from nAudio seem really odd, when I replay it sounds perfect but graphing the buffer showed a picture that looked mostly like noise. It took me a while but I think I've made some headway but I'm a little stuck.
The float array that comes out has a block align of 8, so 4 floats per sample (I'm recording at 16bit so one float should easily hold this. However there are 2 and often 3 (for load) floats provided per sample. I ended up graphing it - Charts of Data. The top picture is the closest I can get to reconstructing the wave, the bottom is the wave as recorded and the middle is a chart of the raw data.
It seems to me that each float is simply holding a byte value but I'm very confused as to the first value which appears to be some kind of scaling factor.
Before I go into to much detail on what I've found I might just leave it at that with the hope Mark will know exactly how/why I am seeing this.
My current best attempt to decode this data is to convert the numbers to bytes then left shift them together which provides the top chart of the attached. I'm fairly sure that there is more to it however.
OK so after a bit more tweaking I figured out that the float array was in fact an array of bytes from floats. Not sure if that makes sense, each "float" in the 4 floats per sample was raw bits that made up floats.
In the end this made it incredibly easy to process the buffer into an array of floats as follows;
_samplesToProcess = floatsIn.Length / WaveFormat.BlockAlign * WaveFormat.Channels;
if (_rawFloatsOut.Length < _samplesToProcess)
_rawFloatsOut = new float[_samplesToProcess];
Buffer.BlockCopy(floatsIn, 0, _rawFloatsOut, 0, floatsIn.Length);
BufferProcessor(_rawFloatsOut);
Hi I'm a noob in audio related coding and I'm working in a pitch tracking DLL that I will use to try to create a sort of open-source version of the video-game Rocksmith as a learning experience.
So far I have managed to get the FFT to work so I can detect pitch frequency (Hz) then by using an algorithm and the table below I can manage to determine the octave (2th to 6th) and the note (C to B) for played note.
The next step is to detect the string so I can determine the fret.
I've been thinking about it and in theory I can work with this, I will know when the user is playing the right note but the game could be "Hack" because by just using the Hz the game is not able to detect if a note is played in the right string. For example 5th string + 1th fret = C4 261.63Hz is equals to 6th string + 5th fret = C4 261.63Hz.
The chances of having an user playing a note in the wrong string and getting it right is low, but I think it would be really good to know the string so I can provide to the users some error feedback when they play the wrong string (Like you should go a string up or down).
Do you know what can I do to detect the string? Thanks in advance :)
[edit]
The guitar and strings that we are using affect the timbre so analyzing the timbre seems to not be a easy way of detecting strings:
"Variations in timbre on your guitar are produced by an enormous number of factors from pickup design and position, the natural resonances and damping in your guitar due to the wood used (that's a different sort of timber!) and its construction and shape, the gauge and age of your strings, your playing technique, where you fret and pluck the string, and so on."
This might be a little bit late because the post is one years old. But here's a solution, which I found out after long research for pitch detecting a guitar.
THIS IS WHY FFT DOESN'T WORK :
You cannot use FFT since the result gives you a linear array, and the sound is calculated logarithmically (exponential distance between notes). Plus, FFT gives you an array of bins in which your frequency COULD BE, it doesnt give you the precise result.
THIS IS WHAT I SUGGEST :
Use dywapitchtrack. it's a library that uses a wavelet algorythm, which works directly on your wave instead of calculating large bins like FFT.
description:
The dywapitchtrack is based on a custom-tailored algorithm which is of very high quality:
both very accurate (precision < 0.05 semitones), very low latency (< 23 ms) and
very low error rate. It has been thoroughly tested on human voice.
It can best be described as a dynamic wavelet algorithm (dywa):
DOWNLOAD : https://github.com/inniyah/sndpeek/tree/master/src/dywapitchtrack
USE(C++):
put the .c and .h where you need it and import it in your project
include the header file
//Create a dywapitchtracker Object
dywapitchtracker pitchtracker;
//Initialise the object with this function
dywapitch_inittracking(&pitchtracker);
When your buffer is full (buffer needs to be at 44100 resolution and power of 2 of length, mine is 2048):
//use this function with your buffer
double thePitch = dywapitch_computepitch(&pitchtracker, yourBuffer, 0, 2048);
And voilà, thePitch contains precisely what you need. (feel free to ask question if something is unclear)
An simple FFT peak estimator is not a good guitar pitch detector/estimator, due to many potentially strong overtones. There exist more robust pitch estimation algorithms (search stackoverflow and DSP.stackexchange). But if you require the players to pre-characterize each string on their individual instruments, both open and fretted, before starting the game, an FFT fingerprint of those characterizations might be able to differentiate the same note played on different strings on some guitars. The thicker strings will give off slightly different ratios of energy in some of the higher overtones, as well as different amounts of slight inharmonicity.
The other answers seem to suggest a simple pitch detection method. However, it is something you will have to research.
Specifically, compare the overtones of 5th string 1st fret to sixth string 5th fret. that is, only look at 261.63*2, 261.63*3, *4, etc. Also, try looking at 261.63*0.5. Compare the amplitudes of the two signals at these freqs. There might be a pattern that could be detected.
I was looking for satisfactory and safe workaround to my double precision issue specified to this problem:
This program tries to find how many small circle can fit into a large circle. It fills the large circle and then culls those that intersect the large circumference. using this formula:
distance_small_pos_from_center + small_radius < big_radius
All calculations were in double, except for screen output on WinForms which takes int for coords.
The above image shows the result of the culling. You can see that it is not symmetric when it should really be because the constraint is that there must be one small circle exactly in the center. I step through the code and find that this is because some calculations yield, for example,
99.9999999 < 100
This answer C++ double precision and rounding off says we should use all the precision available, but in this case, I had to do a Math.Round(distance_small_pos_from_center + small_radius, 3) using 3 arbitarily.
The result of the culling differs very much without Math.Round. In retrospect, this is one kind of bug that is hard to detect if I had not drawn it out. Maybe I did something wrong, or didn't understand doubles as much as I thought I had.
So, anyone has solutions or tips to avoid this kind of problem?
Sorry for not beeing able to provide a complete answer to your question, but i have no time for that right now. But when you compare floats, compare them with a "tolerance" since a float is not exact.
EDIT: modified with abs() in case you don't know which is big and small, as pointed out by Hans Kesting
Ie, do something like if(abs(big_radius - distance_small_pos_from_center) < epsilon) where epsilon is your tolerance, selected with consideration to how "inexact" the floats will be in the range where you are working..
For more precise information see:
http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm
http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html
http://www.cplusplus.com/forum/articles/3638/
Use System.Decimal:
http://msdn.microsoft.com/en-us/library/system.decimal.aspx
I'm working on an algorithm to find peaks in a List object. I'd thought up what I thought was a good (or good enough) algorithm for doing this by looking at a point and it's neighbors and, if it was a peak, adding it to the results list. However, given some recent results, I don't think this method works as well as I'd initially hoped. (I've included the code I'm currently using, and hope to replace, below). I've done a little work with LabView before and I know that the way their module finds peaks/valleys works for what I need to do. I did some research into how LabView does this and found this:
"This Peak Detector VI is based on an algorithm that fits a quadratic polynomial to sequential groups of data points. The number of data points used in the fit is specified by width.
For each peak or valley, the quadratic fit is tested against the threshold. Peaks with heights lower than the threshold or valleys with troughs higher than the threshold are ignored. Peaks and valleys are detected only after the VI processes approximately width/2 data points beyond the location of the peak or valley. This delay has implications only for real-time processing."
Okay, so now I've been trying to do something similar in C#, however, in all my searching it seems that fitting a quadratic polynomial to data is certainly not trivial. I'd think that this problem would be one explored many, many times, but I've been unsuccessful getting a algorithm that does this well or finding a library to do it with.
Any help with this problem is greatly appreciated. Thanks.
Original/Current Code:
public static List<double> FindPeaks(List<double> values, double rangeOfPeaks)
{
List<double> peaks = new List<double>();
int checksOnEachSide = (int)Math.Floor(rangeOfPeaks / 2);
for (int i = checksOnEachSide; i < values.Count - checksOnEachSide; i++)
{
double current = values[i];
IEnumerable<double> window = values;
if (i > checksOnEachSide)
window = window.Skip(i - checksOnEachSide);
window = window.Take((int)rangeOfPeaks);
if (current == window.Max())
peaks.Add(current);
}
return peaks;
}
I have used Math.NET for matrix operations like this in c#. It has all the tools you might need for least squares problems such as QR decomposition or SVD. For a general overview of how to apply them I think wikipedia does quite a good job.
Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.
I'd like to write a simple C# application to monitor the line-in audio and give me the current (well, the rolling average) beats per minute.
I've seen this gamedev article, and that was absolutely no help. I went through and tried to implement what he was doing but it just wasn't working.
I know there have to be tons of solutions for this, because lots of DJ software does it, but I'm not having any luck in finding any open-source library or instructions on doing it myself.
Calculate a powerspectrum with a sliding window FFT:
Take 1024 samples:
double[] signal = stream.Take(1024);
Feed it to an FFT algorithm:
double[] real = new double[signal.Length];
double[] imag = new double[signal.Length);
FFT(signal, out real, out imag);
You will get a real part and an imaginary part. Do NOT throw away the imaginary part. Do the same to the real part as the imaginary. While it is true that the imaginary part is pi / 2 out of phase with the real, it still contains 50% of the spectrum information.
EDIT:
Calculate the power as opposed to the amplitude so that you have a high number when it is loud and close to zero when it is quiet:
for (i=0; i < real.Length; i++) real[i] = real[i] * real[i];
Similarly for the imaginary part.
for (i=0; i < imag.Length; i++) imag[i] = imag[i] * imag[i];
Now you have a power spectrum for the last 1024 samples. Where the first part of the spectrum is the low frequencies and the last part of the spectrum is the high
frequencies.
If you want to find BPM in popular music you should probably focus on the bass. You can pick up the bass intensity by summing the lower part of the power spectrum. Which numbers to use depends on the sampling frequency:
double bassIntensity = 0;
for (i=8; i < 96; i++) bassIntensity += real[i];
Now do the same again but move the window 256 samples before you calculate a new spectrum. Now you end up with calculating the bassIntensity for every 256 samples.
This is a good input for your BPM analysis. When the bass is quiet you do not have a beat and when it is loud you have a beat.
Good luck!
There's an excellent project called Dancing Monkeys, which procedurally generates DDR dance steps from music. A large part of what it does is based on (necessarily very accurate) beat analysis, and their project paper goes into much detail describing the various beat detection algorithms and their suitability to the task. They include references to the original papers for each of the algorithms. They've also published the matlab code for their solution. I'm sure that between those you can find what you need.
It's all available here: http://monket.net/dancing-monkeys-v2/Main_Page
Not that I have a clue how to implement this, but from an audio engineering perspective you'd need to filter first. Bass drum hits would be the first to check. A low pass filter that gives you anything under about 200Hz should give you a pretty clear picture of the bass drum. A gate might also be necessary to cleanup any clutter from other instruments with harmonics that low.
The next to check would be snare hits. You'd have to EQ this one. The "crack" from a snare is around 1.5kHz from memory, but you'd need to definitely gate this one.
The next challenge would be to work out an algorithm for funky beats. How would you programatically find beat 1? I guess you'd keep track of previous beats and use a pattern matching something-or-other. So, you'd probably need a few bars to accurately find the beat. Then there's timing issues like 4/4, 3/4, 6/8, wow, I can't imagine what would be required to do this accurately! I'm sure it'd be worth some serious money to audio hardware/software companies.
This is by no means an easy problem. I'll try to give you an overview only.
What you could do is something like the following:
Compute the average (root-mean-square) loudness of the signal over blocks of, say, 5 milliseconds. (Having never done this before, I don't know what a good block size would be.)
Take the Fourier transform of the "blocked" signal, using the FFT algorithm.
Find the component in the transformed signal that has the largest magnitude.
A Fourier transform is basically a way of computing the strength of all frequencies present in the signal. If you do that over the "blocked" signal, the frequency of the beat will hopefully be the strongest one.
Maybe you need to apply a filter first, to focus on specific frequencies (like the bass) that usually contain the most information about the BPM.
I found this library which seem to have a pretty solid implementation for detecting Beats per Minute.
https://github.com/owoudenberg/soundtouch.net
It's based on http://www.surina.net/soundtouch/index.html which is used in quite a few DJ projects http://www.surina.net/soundtouch/applications.html
First of all, what Hallgrim is producing is not the power spectral density function. Statistical periodicities in any signal can be brought out through an autocorrelation function. The fourier transform of the autocorrelation signal is the power spectral density. Dominant peaks in the PSD other than at 0 Hz will correspond to the effective periodicity in the signal (in Hz)...
The easy way to do it is to have the user tap a button in rhythm with the beat, and count the number of taps divided by the time.
I'd recommend checking out the BASS audio library and the BASS.NET wrapper. It has a built in BPMCounter class.
Details for this specific function can be found at
http://bass.radio42.com/help/html/0833aa5a-3be9-037c-66f2-9adfd42a8512.htm.