I'm trying to design an auto-focus system for a low cost USB microscope. I have been developing the hardware side with a precision PAP motor that is able to adjust the focus knob in the microscope, and now I'm in the hard part.
I have been thinking about how to implement the software. The hardware have two USB ports, one for the microscope camera and another for the motor. My initial idea is to write an application in C# that is able to get the image from the microscope and to move the motor forth and backwards, so far so good :)
Now I need a bit of help with the auto-focus, how to implement it? There is any good algorithm for this? Or maybe an image processing library that will help my in my task?
I have been googleling but with no success... I'll appreciate any help/idea/recommendation!
Many thanks :)
EDIT: Thanks guys for your answers, i'll try all the options and get back here with the results (or maybe more questions).
The most important piece is code which tells you how much out of focus the image is. Since an unfocused image loses high frequency data I'd try something like the following:
long CalculateFocusQuality(byte[,] pixels)
{
long sum = 0;
for(int y = 0; y<height-1; y++)
for(int x=0; x<width-1; x++)
{
sum += Square(pixels[x+1, y] - pixels[x, y]);
sum += Square(pixels[x, y] - pixels[x, y+1]);
}
return sum;
}
int Square(int x)
{
return x*x;
}
This algorithm doesn't work well if the image is noisy. In that case you could to downsample it, or use a more complex algorithm.
Or another idea is calculating the variation of the pixel values:
long CalculateFocusQuality(byte[,] pixels)
{
long sum = 0;
long sumOfSquares = 0;
for(int y=0; y<height; y++)
for(int x=0; x<width; x++)
{
byte pixel=pixels[x,y];
sum+=pixel;
sumofSquares+=pixel*pixel;
}
return sumOfSquares*width*height - sum*sum;
}
These functions work on monochromatic images, for RGB images just sum the values for the channels.
Using this function change the focus trying to maximize CalculateFocusQuality. Increase the stepsize if several attempts in a row improved the quality, and decrease it and reverse the direction if the step reduced the quality.
Autofocusing a microcoscope is a long standing topic in optical research.
You can learn a bit about the involved algorithms here.
The problems involved are not only how to meassure defocus, but also how to move the optical axis in an optimal way, and how to correct algorithmically the residual aberrations.
HTH!
just some of my experiences trying to solve similar task. On my system a 200x magnificatin is used. Stepper resolutin in Z-direction 0.001um.
The problems I've faced:
-Shaking. The image on theoretically better position could be evaluated worse because of suddenly shaking. As the API of my system didn't allow to move z-axix and make images in parallel, I had to move in steps and capture sequenttially. Each move-stop caused shaking. Interestingly, the shaking were more severe while moving down than moving up.
-Mechanical imprecision. Making a scan and moving to theoretically best position may bear an error, because stepper-position in controller may be not the same as the mechanical position.
-Exposure: Depending on the application, the brightness of the image may vary, so that exposure should be adjusted. Depending on focus-evaluation algorithm (whether brightness is involved in the calculation or not) the exposure may be required to be fixed. That results in the chicken-egg problem - how to setup exposure, if image brightness is unknown and how to focus, if required exposure is unknown.
Finally, to avoid mechanical problems I've (re)stored best image found while focusing and returned it at the end.
Concerning the algorithm for focus-value, the best was looking for edges combined with entire number of colors (histogram width). But of cause, it depends on the type of image you process.
Regards,
Valentin Heinitz
There's some information on Wikipedia
Technically it can be implemented as
high pass filter and some system which
conscientiously moves lens around the
point where filter output is highest.
Digital processing is not required
Also, 5 out of the first 6 matches I get from Googling for "autofocus algorithm" seem to have relevant and useful information (although in one or two cases the full details of papers requires payment)
Related
I'm working on an algorithm to find peaks in a List object. I'd thought up what I thought was a good (or good enough) algorithm for doing this by looking at a point and it's neighbors and, if it was a peak, adding it to the results list. However, given some recent results, I don't think this method works as well as I'd initially hoped. (I've included the code I'm currently using, and hope to replace, below). I've done a little work with LabView before and I know that the way their module finds peaks/valleys works for what I need to do. I did some research into how LabView does this and found this:
"This Peak Detector VI is based on an algorithm that fits a quadratic polynomial to sequential groups of data points. The number of data points used in the fit is specified by width.
For each peak or valley, the quadratic fit is tested against the threshold. Peaks with heights lower than the threshold or valleys with troughs higher than the threshold are ignored. Peaks and valleys are detected only after the VI processes approximately width/2 data points beyond the location of the peak or valley. This delay has implications only for real-time processing."
Okay, so now I've been trying to do something similar in C#, however, in all my searching it seems that fitting a quadratic polynomial to data is certainly not trivial. I'd think that this problem would be one explored many, many times, but I've been unsuccessful getting a algorithm that does this well or finding a library to do it with.
Any help with this problem is greatly appreciated. Thanks.
Original/Current Code:
public static List<double> FindPeaks(List<double> values, double rangeOfPeaks)
{
List<double> peaks = new List<double>();
int checksOnEachSide = (int)Math.Floor(rangeOfPeaks / 2);
for (int i = checksOnEachSide; i < values.Count - checksOnEachSide; i++)
{
double current = values[i];
IEnumerable<double> window = values;
if (i > checksOnEachSide)
window = window.Skip(i - checksOnEachSide);
window = window.Take((int)rangeOfPeaks);
if (current == window.Max())
peaks.Add(current);
}
return peaks;
}
I have used Math.NET for matrix operations like this in c#. It has all the tools you might need for least squares problems such as QR decomposition or SVD. For a general overview of how to apply them I think wikipedia does quite a good job.
Hello
I'm exploring the audio possibilities of the WP7 platform and the first stumble I've had is trying to implement a FFT using the Cooley-Tukey method. The result of that is that the spectrogram shows 4 identical images in this order: one normal, one reversed, one normal, one reversed.
The code was taken from another C# project (for desktop), the implementation and all variables seem in place with the algorithm.
So I can see two problems right away: reduced resolution and CPU wasted to generate four identical spectrograms.
Given a sample size of 1600 (could be 2048) I know have only 512 usable frequency information which leaves me with a 15Hz resolution for an 8kHz frequency span. Not bad, but not so good either.
Should I just give up on the code and use NAudio? I cannot seem to have an explanation why the spectrum is quadrupled, input data is ok, algorithm seems ok.
This sounds correct. You have 2 mirrors, I can only assume that one is the Real part and the other is the Image part. This is standard FFT.
From the real and image you can compute the magnitude or amplitude of each harmonic which is more common or compute the angle or phase shift of each harmonic which is less common.
Gilad.
I switched to NAudio and now the FFT works. However I might have found the cause (I probably won't try to test again): when I was constructing an array of double to feed into the FFT function, I did something like:
for (int i = 0; i < buffer.Length; i+= sizeof(short))
{
samples[i] = ReadSample(buffer, i);
}
For reference, 'samples' is the double[] input to fft, ReadSample is something that takes care of little/big endian. Can't remember right now how the code was, but it was skipping every odd sample.
My math knowledge has never been great but I'm thinking this induces some aliasing patterns which might in the end produce the effect I experienced.
Anyway, problem worked around, but thanks for your input and if you can still explain the phenomenon I am grateful.
I am trying to figure out a way of getting Sikuli's image recognition to use within C#. I don't want to use Sikuli itself because its scripting language is a little slow, and because I really don't want to introduce a java bridge in the middle of my .NET C# app.
So, I have a bitmap which represents an area of my screen (I will call this region BUTTON1). The screen layout may have changed slightly, or the screen may have been moved on the desktop -- so I can't use a direct position. I have to first find where the current position of BUTTON1 is within the live screen. (I tried to post pictures of this, but I guess I can't because I am a new user... I hope the description makes it clear...)
I think that Sikuli is using OpenCV under the covers. Since it is open source, I guess I could reverse engineer it, and figure out how to do what they are doing in OpenCV, implementing it in Emgu.CV instead -- but my Java isn't very strong.
I looked for examples showing this, but all of the examples are either extremely simple (ie, how to recognize a stop sign) or very complex (ie how to do facial recognition)... and maybe I am just dense, but I can't seem to make the jump in logic of how to do this.
Also I worry that all of the various image manipulation routines are actually processor intensive, and I really want this as lightweight as possible (in reality I might have lots of buttons and fields I am trying to find on a screen...)
So, the way I am thinking about doing this instead is:
A) Convert the bitmaps to byte arrays and do brute force search. (I know how to do that part). And then
B) Use the byte array position that I found to calculate its screen position (I'm really not completely sure how I do this) instead of using the image processing stuff.
Is that completely crazy? Does anyone have a simple example of how one could use Aforge.Net or Emgu.CV to do this? (Or how to flesh out step B above...?)
Thanks!
Generally speaking, it sounds like you want basic object recognition. I don't have any experience with SIKULI, but there are a number of ways to do object recognition (Edge based template matching, etc.). That being said you might be able to go with just straight histogram matching.
http://www.codeproject.com/KB/GDI-plus/Image_Processing_Lab.aspx
That page should show you how to use AForge.net to get the histogram of an image. You would just do a brute force search using something like this:
Bitmap ImageSearchingWithin=new Bitmap("Location of image"); //or just load from a screenshot or whatever
for (int x = 0; x < ImageSearchingWithin.Width - WidthOfImageSearchingFor; ++x)
{
for (int y = 0; y < ImageSearchingWithin.Height - HeightOfImageSearchingFor; ++y)
{
Bitmap MySmallViewOfImage = ImageSearchingWithin.Clone(new Rectangle(x, y, WidthOfImageSearchingFor, HeightOfImageSearchingFor), System.Drawing.Imaging.PixelFormat.Format24bppRgb);
}
}
And then compare the newly created bitmap's histogram to the one that you calculated of the original image (whatever area is the closest in terms of matching is what you would select as being the region of BUTTON1). It's not the most elegant solution but it might work for your needs. Otherwise you get into more difficult techniques (of course I could be forgetting something at the moment that might be simpler).
((Answer selected - see Edit 5 below.))
I need to write a simple pink-noise generator in C#. The problem is, I've never done any audio work before, so I don't know how to interact with the sound card, etc. I do know that I want to stay away from using DirectX, mostly because I don't want to download a massive SDK just for this tiny project.
So I have two problems:
How do I generate Pink Noise?
How do I stream it to the sound card?
Edit: I really want to make a pink noise generator... I'm aware there are other ways to solve the root problem. =)
Edit 2: Our firewall blocks streaming audio and video - otherwise I'd just go to www.simplynoise.com as suggested in the comments. :(
Edit 3: I've got the generation of white-noise down, as well as sending output to the sound card - now all I need to know is how to turn the white-noise into pink noise. Oh - and I don't want to loop a wav file because every application I've tried to use for looping ends up with a tiny little break in between loops, which is jarring enough to have prompted me in this direction in the first place...
Edit 4: ... I'm surprised so many people have jumped in to very explicitly not answer a question. I probably would have gotten a better response if I lied about why I need pink noise... This question is more about how to generate and stream data to the sound card than it is about what sort of headphones I should be using. To that end I've edited out the background details - you can read about it in the edits...
Edit 5: I've selected Paul's answer below because the link he provided gave me the formula to convert white noise (which is easily generated via the random number generator) into pink noise. In addition to this, I used Ianier Munoz's CodeProject entry "Programming Audio Effects in C#" to learn how to generate, modify, and output sound data to the sound card. Thank you guys for your help. =)
Maybe you can convert the C/C++ code here to C#:
http://www.firstpr.com.au/dsp/pink-noise/
The easiest way to get sound to the sound card is to generate a wav (spit out some hardcoded headers and then sample data). Then you can play the .wav file.
Pink noise is just white noise put through a -3dB/octave LPF. You can generate white noise using rand() (or any function that generates uniformly random numbers).
Streaming stuff to the soundcard is reasonably trivial, as long as you have Google handy. If you choose to avoid DirectX, consider using PortAudio or ASIO for interfacing with the soundcard... although I think you're gonna have to use C++ or C.
Other than that, why waste CPU time generating it? Loop a damn WAV file!
bit late to this i realise, but anyone coming across it for answers should know that pink noise is white noise with -3dB/octave, not -6 as stated above, which is actually brown noise.
Here's a very simple way to create pink noise, which just sums lots of waves spaced logarithmically apart, together! It may be too slow for your purposes if you want the sound created in realtime, but further optimization is surely possible (e.g: a faster cosine function).
The functions outputs a double array with values from -1 to 1. This represents the lowest and highest points in the waveform respectively.
The quality parameter represents the number of waves produced to make the sound. I find 5000 waves (about 40 intervals per semitone) is just about the threshold where I can't detect any noticeable improvement with higher values, but to be on the safe side, you could (optionally) increase this to about 10,000 waves or higher. Also, according to Wikipedia, 20 hertz is around the lower limit of human perception in terms of what we can hear, but you can change this too if you want.
Note the sound gets quieter with a higher quality value due to technical reasons, so you may (optionally) want to adjust the volume via the volumeAdjust parameter.
public double[] createPinkNoise(double seconds, int quality=5000, double lowestFrequency=20, double highestFrequency = 20000, double volumeAdjust=1.0)
{
long samples = (long)(44100 * seconds);
double[] d = new double[samples];
double[] offsets = new double[samples];
double lowestWavelength = highestFrequency / lowestFrequency;
Random r = new Random();
for (int j = 0; j < quality; j++)
{
double wavelength = Math.Pow(lowestWavelength, (j * 1.0) / quality) * 44100 / highestFrequency;
double offset = r.NextDouble() * Math.PI*2; // Important offset is needed, as otherwise all the waves will be almost in phase, and this will ruin the effect!
for (long i = 0; i < samples; i++)
{
d[i] += Math.Cos(i * Math.PI * 2 / wavelength + offset) / quality * volumeAdjust;
}
}
return d;
}
Here's is an example of what the playback thread looks like. I'm using DirectSound to create a SecondaryBuffer where the samples are written. As you can see it's pretty straightforward:
/// <summary>
/// Thread in charge of feeding the playback buffer.
/// </summary>
private void playbackThreadFn()
{
// Begin playing the sound buffer.
m_playbackBuffer.Play( 0, BufferPlayFlags.Looping );
// Change playing state.
IsPlaying = true;
// Playback loop.
while( IsPlaying )
{
// Suspend thread until the playback cursor steps into a trap...
m_trapEvent.WaitOne();
// ...read audio from the input stream... (In this case from your pink noise buffer)
Input.Collect( m_target, m_target.Length );
// ...calculate the next writing position...
var writePosition = m_traps[ ((1 & m_pullCounter++) != 0) ? 0 : 1 ].Offset;
// ...and copy audio to the device buffer.
m_playbackBuffer.Write( writePosition, m_deviceBuffer, LockFlag.None );
}
// Stop playback.
m_playbackBuffer.Stop();
}
If you need more details on how it works I'll be glad to help.
If you're on Linux, you can use SOX (you may have it already, try the play command).
play -t sl - synth 3 pinknoise band -n 1200 200 tremolo .1 40 < /dev/zero
As a quick and dirty way to do it, how about just looping a pink noise wav in your audio player? (Yes, I know part of the fun is to make it yourself....)
What about an .mp3 sample of Pink Noise on repeat?
You could use Audacity to generate as much pink noise as you want, and then repeat it.
Or you could dig into the source code and see how Audacity does the pink noise generation.
I can't speak about C#, but you might be better off with some good noise canceling headphones and your favorite mp3's.
Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.
I'd like to write a simple C# application to monitor the line-in audio and give me the current (well, the rolling average) beats per minute.
I've seen this gamedev article, and that was absolutely no help. I went through and tried to implement what he was doing but it just wasn't working.
I know there have to be tons of solutions for this, because lots of DJ software does it, but I'm not having any luck in finding any open-source library or instructions on doing it myself.
Calculate a powerspectrum with a sliding window FFT:
Take 1024 samples:
double[] signal = stream.Take(1024);
Feed it to an FFT algorithm:
double[] real = new double[signal.Length];
double[] imag = new double[signal.Length);
FFT(signal, out real, out imag);
You will get a real part and an imaginary part. Do NOT throw away the imaginary part. Do the same to the real part as the imaginary. While it is true that the imaginary part is pi / 2 out of phase with the real, it still contains 50% of the spectrum information.
EDIT:
Calculate the power as opposed to the amplitude so that you have a high number when it is loud and close to zero when it is quiet:
for (i=0; i < real.Length; i++) real[i] = real[i] * real[i];
Similarly for the imaginary part.
for (i=0; i < imag.Length; i++) imag[i] = imag[i] * imag[i];
Now you have a power spectrum for the last 1024 samples. Where the first part of the spectrum is the low frequencies and the last part of the spectrum is the high
frequencies.
If you want to find BPM in popular music you should probably focus on the bass. You can pick up the bass intensity by summing the lower part of the power spectrum. Which numbers to use depends on the sampling frequency:
double bassIntensity = 0;
for (i=8; i < 96; i++) bassIntensity += real[i];
Now do the same again but move the window 256 samples before you calculate a new spectrum. Now you end up with calculating the bassIntensity for every 256 samples.
This is a good input for your BPM analysis. When the bass is quiet you do not have a beat and when it is loud you have a beat.
Good luck!
There's an excellent project called Dancing Monkeys, which procedurally generates DDR dance steps from music. A large part of what it does is based on (necessarily very accurate) beat analysis, and their project paper goes into much detail describing the various beat detection algorithms and their suitability to the task. They include references to the original papers for each of the algorithms. They've also published the matlab code for their solution. I'm sure that between those you can find what you need.
It's all available here: http://monket.net/dancing-monkeys-v2/Main_Page
Not that I have a clue how to implement this, but from an audio engineering perspective you'd need to filter first. Bass drum hits would be the first to check. A low pass filter that gives you anything under about 200Hz should give you a pretty clear picture of the bass drum. A gate might also be necessary to cleanup any clutter from other instruments with harmonics that low.
The next to check would be snare hits. You'd have to EQ this one. The "crack" from a snare is around 1.5kHz from memory, but you'd need to definitely gate this one.
The next challenge would be to work out an algorithm for funky beats. How would you programatically find beat 1? I guess you'd keep track of previous beats and use a pattern matching something-or-other. So, you'd probably need a few bars to accurately find the beat. Then there's timing issues like 4/4, 3/4, 6/8, wow, I can't imagine what would be required to do this accurately! I'm sure it'd be worth some serious money to audio hardware/software companies.
This is by no means an easy problem. I'll try to give you an overview only.
What you could do is something like the following:
Compute the average (root-mean-square) loudness of the signal over blocks of, say, 5 milliseconds. (Having never done this before, I don't know what a good block size would be.)
Take the Fourier transform of the "blocked" signal, using the FFT algorithm.
Find the component in the transformed signal that has the largest magnitude.
A Fourier transform is basically a way of computing the strength of all frequencies present in the signal. If you do that over the "blocked" signal, the frequency of the beat will hopefully be the strongest one.
Maybe you need to apply a filter first, to focus on specific frequencies (like the bass) that usually contain the most information about the BPM.
I found this library which seem to have a pretty solid implementation for detecting Beats per Minute.
https://github.com/owoudenberg/soundtouch.net
It's based on http://www.surina.net/soundtouch/index.html which is used in quite a few DJ projects http://www.surina.net/soundtouch/applications.html
First of all, what Hallgrim is producing is not the power spectral density function. Statistical periodicities in any signal can be brought out through an autocorrelation function. The fourier transform of the autocorrelation signal is the power spectral density. Dominant peaks in the PSD other than at 0 Hz will correspond to the effective periodicity in the signal (in Hz)...
The easy way to do it is to have the user tap a button in rhythm with the beat, and count the number of taps divided by the time.
I'd recommend checking out the BASS audio library and the BASS.NET wrapper. It has a built in BPMCounter class.
Details for this specific function can be found at
http://bass.radio42.com/help/html/0833aa5a-3be9-037c-66f2-9adfd42a8512.htm.