We Keep Coding

How to avoid audio glitches whilst switching volume of an Audiosource each Update()-Frame?

I'm developing an audiotool that plays 64 Audiosources simultaneously. Therefore I created four arrays containing 16 Audiosources each. Each array of Audiosources is routed to its own Mixer. Furthermore, two mixer output to the left channel, two to the right. My DSP Buffer Size is set to Best Performance, meaning 1024 samples and there are enough real / virtual voices available.
In the beginning, 60 Audiosources are set to Volume = 0, while four of them are running with Volume = 0.5. Each Update()-Frame, I set the Volume of those playing with 0.5 to zero, therefore setting four new audiosources that were zero before to 0.5.
Something like this:
void SwitchSources()
{
noseposInd++;
if (noseposInd > 15) noseposInd = 0;
audioSources_Lm[noseposIndTemp].volume = 0.0f;
audioSources_Ln[noseposIndTemp].volume = 0.0f;
audioSources_Rm[noseposIndTemp].volume = 0.0f;
audioSources_Rn[noseposIndTemp].volume = 0.0f;
audioSources_Lm[noseposInd].volume = 0.5f;
audioSources_Ln[noseposInd].volume = 0.5f;
audioSources_Rm[noseposInd].volume = 0.5f;
audioSources_Rn[noseposInd].volume = 0.5f;
noseposIndTemp = noseposInd;
}
For test purposes, I loaded a rectangle signal with f = 2Hz (results in an audible click per second) into each Audiosource. Recording my output with Audacity results in something that can be seen on the attached photo:
It seems that the buffer of one of the four signals is not written to the output because the amplitude regarding a positive or negative pulse is just half. The width of the "notches" is exactly one blocklength. Meaning 1024 samples with a samplerate of 44.1kHz, so that there is no output for about 23ms.
Increasing the rate of changing the volume also increases the occurences of notches / time outs or however this can be called. Has anyone had the same problem or can help out with some knowledge about how the Update()-Method and the audio-block-writing of the mixers interfere?
Thanks in advance!

reduce CPU overhead while proccessing Video Stream

I am developing C# WPF Auto Number Plate Recognition Using an OCR.
The Flow is, i am getting a pictures from a video stream MJPEG and this images should be passed to the OCR to get the plate number and other details.
The problem is : the Video stream is producing about 30 Frame/second and the CPU can't handle this much of processing also it will take around 1 Sec to process 1 frame, Also when i will get many frames on the Queue the CPU will be 70% used (Intel I7 4th G).
Can anyone suggest solution and better implementation.
//This is the queue where it will hold the frames
// produced from the video streaming(video_Newfram1)
private readonly Queue<byte[]> _anpr1Produces = new Queue<byte[]>();
//I am using AForg.Video to read the MJPEG Streaming
//this event will be triggered for every frame
private void video_NewFrame1(object sender, NewFrameEventArgs eventArgs)
{
var frameDataAnpr = new Bitmap(eventArgs.Frame);
AnprCam1.Source = GetBitmapimage(frameDataAnpr);
//add current fram to the queue
_anpr1Produces.Enqueue(imgByteAnpr);
//this worker is the consumer that will
//take the frames from the queue to the OCR processing
if (!_workerAnpr1.IsBusy)
{
_workerAnpr1.RunWorkerAsync(imgByteAnpr);
}
}
//This is the consumer, it will take the frames from the queue to the OCR
private void WorkerAnpr1_DoWork(object sender, DoWorkEventArgs e)
{
while (true)
{
if (_anpr1Produces.Count <= 0) continue;
BgWorker1(_anpr1Produces.Dequeue());
}
}
//This method will process the frames that sent from the consumer
private void BgWorker1(byte[] imageByteAnpr)
{
var anpr = new cmAnpr("default");
var objgxImage = new gxImage("default");
if (imageByteAnpr != null)
{
objgxImage.LoadFromMem(imageByteAnpr, 1);
if (anpr.FindFirst(objgxImage) && anpr.GetConfidence() >= Configs.ConfidanceLevel)
{
var vehicleNumber = anpr.GetText();
var vehicleType = anpr.GetType().ToString();
if (vehicleType == "0") return;
var imagename = string.Format("{0:yyyy_MMM_dd_HHmmssfff}", currentDateTime) + "-1-" +
vehicleNumber + ".png";
//this task will run async to do the rest of the process which is saving the vehicle image, getting vehicle color, storing to the database ... etc
var tsk = ProcessVehicle("1", vehicleType, vehicleNumber, imageByteAnpr, imagename, currentDateTime, anpr, _anpr1Produces);
}
else
{
GC.Collect();
}
}
}

What you should do is this:
First, figure out if a frame is worth processing or not. If you're using a compressed video stream, you can usually quickly read the frame's compressed size. It stores the difference between the current frame and the previous one.
When it's small, not much changed (i.e: no car drove by).
That's a low-tech way to do motion detection, without even having to decode a frame, and it should be extremely fast.
That way, you can probably decide to skip 80% of the frames in a couple of milliseconds.
Once and a while you'll get frames that need processing. Make sure that you can buffer enough frames so that you can keep recording while you're doing your slow processing.
The next thing to do is find a region of interest, and focus on those first. You could do that by simply looking at areas where the color changed, or try to find rectangular shapes.
Finally, one second of processing is SLOW if you need to process 30 fps. You need to make things faster, or you'll have to build up a gigantic buffer, and hope that you'll ever catch up if it's busy on the road.
Make sure to make proper use of multiple cores if they are available, but in the end, knowing which pieces of the image are NOT relevant is the key to faster performance here.

NAudio: Using MixingSampleProvider correctly with VolumeSampleProvider

I have been using NAudio with the
"Fire and Forget Audio Playback with NAudio" tutorial (thank you Mark for this awesome utility!) as written here:
http://mark-dot-net.blogspot.nl/2014/02/fire-and-forget-audio-playback-with.html
I managed to add a VolumeSampleProvider to it, using the MixingSampleProvider as input. However, when I now play two sounds right after each other, the first sound always gets the volume of the second as well, even though the first is already playing.
So my question is: How do I add sounds with an individual volume per sound?
This is what I used:
mixer = new MixingSampleProvider(waveformat);
mixer.ReadFully = true;
volumeProvider = new VolumeSampleProvider(mixer);
panProvider = new PanningSampleProvider(volumeProvider);
outputDevice.Init(panProvider);
outputDevice.Play();

I realized (thanks to itsmatt) that the only way to make this work, is to leave the mixer alone and adjust the panning and volume of each CachedSound individually, before adding it to the mixer. Therefore I needed to rewrite the CachedSoundSampleProvider, using a pan and volume as extra input parameters.
This is the new constructor:
public CachedSoundSampleProvider(CachedSound cachedSound, float volume = 1, float pan = 0)
{
this.cachedSound = cachedSound;
LeftVolume = volume * (0.5f - pan / 2);
RightVolume = volume * (0.5f + pan / 2);
}
And this is the new Read() function:
public int Read(float[] buffer, int offset, int count)
{
long availableSamples = cachedSound.AudioData.Length - position;
long samplesToCopy = Math.Min(availableSamples, count);
int destOffset = offset;
for (int sourceSample = 0; sourceSample < samplesToCopy; sourceSample += 2)
{
float outL = cachedSound.AudioData[position + sourceSample + 0];
float outR = cachedSound.AudioData[position + sourceSample + 1];
buffer[destOffset + 0] = outL * LeftVolume;
buffer[destOffset + 1] = outR * RightVolume;
destOffset += 2;
}
position += samplesToCopy;
return (int)samplesToCopy;
}

I'm not 100% certain of what you are asking and I don't know if you solved this already but here's my take on this.
ISampleProvider objects play the "pass the buck" game to their source ISampleProvider via the Read() method. Eventually, someone does some actual reading of audio bytes. Individual ISampleProvider classes do whatever they do to the bytes.
MixingSampleProvider, for instance, takes N audio sources... those get mixed. When Read() is called, it iterates the audio sources and reads count bytes from each.
Passing it to a VolumeSampleProvider handles all the bytes (from those various sources) as a group... it says:
buffer[offset+n] *= volume;
That's going to adjust the bytes across the board... so every byte gets adjusted in the buffer by the volume multiplier;
The PanningSampleProvider just provides a multiplier to the stereo audio and adjusts the bytes accordingly, doing the same sort of thing as the VolumeSampleProvider.
If you want to individually handle audio source volumes, you need to handle that upstream of the MixingSampleProvider. Essentially, the things that you pass to the MixingSampleProvider need to be able to have their volume adjusted independently.
If you passed a bunch of SampleChannel objects to your MixingSampleProvider... you could accomplish independent volume adjustment. The Samplechannel class incorporates a VolumeSampleProvider object and provides a Volume property that allows one to set the volume on that VolumeSampleProvider object.
SampleChannel also incorporates a MeteringSampleProvider that provides reporting of the maximum sample value during a given period. It raises an event that gives you an array of those values, one per channel.

Using Naudio WaveIn for both performing FFT transformation and writing to disk in real-time (NOT ASIO)

I am currently creating a Winforms application for Windows 8.1, I have been able to perform an FFT on the input data from the devices microphone using ASIO Out, however to be able to use ASIO on my machine I needed to download ASIO4ALL,
This is causing a huge amount of feedback in the microphone and is resulting in very inaccurate frequency readings (to make sure it was the sound itself I wrote a copy to disc to playback),
So to get around this I have been trying to adapt my code to work with Naudio's WaveIn class, however this is returning either no data or NaN for the FFT algorithm (although I can save a recording to disk which plays back with no issues),
I've been trying to fix this for some time now and am sure it is just a silly mistake somewhere, any help would be greatly appreciated!
Below is the code for the "OnDataAvailable" event (where I'm 99% sure I am going wrong):
void OnDataAvailable(object sender, WaveInEventArgs e)
{
if (this.InvokeRequired)
{
this.BeginInvoke(new EventHandler<WaveInEventArgs>(OnDataAvailable), sender, e);
}
else
{
byte[] buffer = e.Buffer;
int bytesRecorded = e.BytesRecorded;
int bufferIncrement = waveIn.WaveFormat.BlockAlign;
for (int index = 0; index < bytesRecorded; index += bufferIncrement)
{
float sample32 = BitConverter.ToSingle(buffer, index);
sampleAggregator.Add(sample32);
}
if (waveFile != null)
{
waveFile.Write(e.Buffer, 0, e.BytesRecorded);
waveFile.Flush();
}
}
}
If any more details and/or code is required please let me know.
waveFile: Name of the file writer
e.Buffer: The buffer containing the recorded data
e.BytesRecorded: The total number of bytes recorded
For reference below is the working code when using the ASIO class:
void asio_DataAvailable(object sender, AsioAudioAvailableEventArgs e)
{
byte[] buf = new byte[e.SamplesPerBuffer * 4];
for (int i = 0; i < e.InputBuffers.Length; i++)
{
Marshal.Copy(e.InputBuffers[i], buf, 0, e.SamplesPerBuffer * 4);
}
for (int i = 0; i < e.SamplesPerBuffer * 4; i++)
{
float sample32 = Convert.ToSingle(buf[i]);
sampleAggregator.Add(sample32);
}
}
EDIT: The samples which are being returned are now accurate after changing the convert statement to Int16 as per the advice on this page, I had some other issues in my code which prevented actual results from being returned originally.
However, the file which is being written to disk is very choppy, I'm sure this is a problem with my laptop and the number of processes which is trying to perform, could anyone please advise a way around this issue?

In the NAudio WPF demo project there is an example of calculating FFTs while playback is happening with a class called SampleAggregator, that stores up blocks of 1024 samples and then performs FFTs on them.
It looks like you are trying to do something similar to this. I suspect the problem is that you are getting 16 bit samples, not 32 bit. Try using BitConverter.ToShort on every pair of bytes.

mWaveInDevice = new WaveIn();
mWaveInDevice.WaveFormat = WaveFormat.**CreateIeeeFloatWaveFormat(44100,2)**;
Set CreateIeeeFloatWaveFormat for WaveFormat, and then you will get right values after fft.

Mediaelement WPF with 2 video one is sometimes freezing

I develop a desktop application using wpf and mediaelement.
There's 2 different mediaelement with all transport fonction play, pause, next/prev frame....
I have 2 problems in my application:
- Sometimes one video is freezing, it's often happen when I'm doing a play and then nextframe or nextframe many times on both video1 and video2.
- When I set the position of the video the image of the video does not always correspond to the position I setted.
Is there anyway to force the video to really refresh to the good position?
Any idea for the freezing staff?
Code for NextFrame:
m_currentMedia.Pause();
m_currentMedia1.Pause();
VideoStatus = VideoStatusEnum.Pause;
VideoStatus1 = VideoStatusEnum.Pause;
SpeedRatio = 0;
SpeedRatio1 = 0;
double NewPos = Math.Round(m_currentMedia.Position.TotalSeconds, 2) + 0.04;
TimeSpan NewPosition = TimeSpan.FromSeconds(NewPos);
double NewPos1 = Math.Round(m_currentMedia1.Position.TotalSeconds, 2) + 0.04;
TimeSpan NewPosition1 = TimeSpan.FromSeconds(NewPos1);
m_currentMedia.Position = NewPosition;
m_currentMedia1.Position = NewPosition1;