I have never used pointers before and now I need to and even though I've read up on them I still have no clue on how to use them!
I am using a third party product (Mitov Signal Lab)and I am using a control that has an event that will allow me to intercept the data. When I enter the event I have access to "args" and one of the statements will return a pointer. Now this points to a buffer of 4096 double values (one a real number and the next an imaginiary number etc). So this is the code:
private void genericComplex1_ProcessData(object Sender, Mitov.SignalLab.ProcessComplexNotifyArgs Args)
{
unsafe
{
IntPtr pointer = Args.InBuffer.Read();
// now what???? I want to get each double value, do some calculations on them
// and return them to the buffer.
}
args.sendOutPutData = true;
}
I'm assuming that i need to assign pointer to Args.InBuffer.Read()
But I've tried some examples I've seen and I just get errors. I'm using visual studio 2010 and i'm compiling with the /unsafe.
Here's an update. There are additional methods such as Args.Inbuffer.Write(), and Args.Inbuffer.Modify() but I still can't figure out how to access the individual components of the buffer.
Ok here are more details. I used the Marshall.copy. Here is the code:
private void genericComplex1_ProcessData(object Sender, Mitov.SignalLab.ProcessComplexNotifyArgs Args)
{
//used to generate a phase corrected Q signal
//equations:
// real = real
// imaginary = (1 + amplitudeimbalance)(real*cos(phaseImbalance) + imaginary*sin(phaseImbalance)
//but amplitude imbalance is already corrected therefore
//imaginary = real * cos(phaseCorrection) + imaginary*sin(phaseCorrection)
double real;
double imaginary;
double newImaginary;
var bufferSize = Convert.ToInt32(Args.InBuffer.GetSize());
Mitov.SignalLab.ComplexBuffer ComplexDataBuffer = new Mitov.SignalLab.ComplexBuffer(bufferSize);
ComplexDataBuffer.Zero();
IntPtr pointer = Args.InBuffer.Read();
IntPtr pointer2 = ComplexDataBuffer.Write();
var data = new double[8192];
Marshal.Copy(pointer, data, 0, 8192);
for (int i = 0; i < 8192; i+=2)
{
data[i] = data[i];
data[i + 1] = data[i] * Math.Cos(phaseCorrection) + data[i+1] * Math.Sin(phaseCorrection);
}
Marshal.Copy(data, 0, pointer2, 8192);
Args.SendOutputData = false;
genericComplex1.SendData(ComplexDataBuffer);
}
Now, this does not work. The reason is it takes too long. I am sampling audio from a sound card at 192,000 samples a second. The event above fires each time there is a 4096 buffer available. Thus every 20 milliseconds a new buffer arrives. This buffer is then sent to a fast fourier transform routine that produces a graph of signal strength vs frequency. The imaginary data is actually the same as the real data but it is phase shifted by 90 degrees. This is how it is in a perfect world. However the phase is never 90 degrees due to imbalances in the signal, different electronics paths, different code and so on. Thus one needs a correction factor and that is what the equations do. Now if I replace the
data[i] = data[i];
data[i + 1] = data[i] * Math.Cos(phaseCorrection) + data[i+1] * Math.Sin(phaseCorrection);
with
data[i] = data[i];
data[i+1] = data[i+1];
it works ok.
So I'm going to need pointers. But will this be substantially faster or will the sin and cosine functions just take too long? Cn I send the data directly to the processor (now thats getting real unsafe!).
You can't access unmanaged data directly in safe context, so you need to do a copy of data to managed container:
using System.Runtime.InteropServices;
// ...
var data = new double[4096];
Marshal.Copy(pointer, data, 0, 4096);
// process data here
And as far as I see it, unsafe context is not required for this part of code.
As for the writing data back it's actually quite the same, but note the parameters order:
// finished processing, copying data back
Marshal.Copy(data, 0, pointer, 4096);
As for your last notes on perfprmance. As I already mentioned, move var data = new double[8192]; to a data class member and initialize it in class constructor, so it's not allocated each time (and allocation is pretty slow). You can do Math.Cos(phaseCorrection) outside of cycle since phaseCorrection is unchanged, same is for Math.Sin(phaseCorrection) of course. Something like this:
Marshal.Copy(pointer, this.data, 0, 8192);
var cosCache = Math.Cos(phaseCorrection);
var sinCache = Math.Sin(phaseCorrection);
for (int i = 0; i < 8192; i+=2)
{
data[i] = data[i];
data[i + 1] = data[i] * cosCache + data[i+1] * sinCache;
}
Marshal.Copy(this.data, 0, pointer2, 8192);
All this should speed up things tremendously.
If you have an IntPtr, you can get a pointer from it. Using your example:
unsafe
{
IntPtr pointer = Args.InBuffer.Read();
// create a pointer to a double
double* pd = (double*)pointer.ToPointer();
// Now you can access *pd
double firstDouble = *pd;
pd++; // moves to the next double in the memory block
double secondDouble = *pd;
// etc.
}
Now, if you want to put data back into the array, it's much the same:
double* pd = (double*)pointer.ToPointer();
*pd = 3.14;
++pd;
*pd = 42.424242;
You need to read and understand Unsafe Code and Pointers. Be very careful with this; you're programming without a safety net here.
Related
My code is designed to get data from a serial device and print its contents to a MS Forms Application. The IDE i use is Visual Studio 2019 - Community.
The device does send a variable size packet. First i have to decode the packet "header" to get crucial information for further processing which is the first packet channel as well as the packet length.
Since the packet does neither contain a line ending, nor a fixed character at the end, the functions SerialPort.ReadTo() and SerialPort.ReadLine() are not useful. Therefore only SerialPort.Read(buf,offset,count) can be used
Since sending rather large packets (512bytes) does take time, I've implemented a function for calculation of a desired wait time, which is defined as (1000ms/baud-rate*(8*byte-count))+100ms
While testing, I've experienced delay, much more than the desired wait times, so implemented a measure function for different parts of the function.
In regular cases (with desired wait times) i except a log to console like this:
Load Header(+122ms) Load Data (+326ms) Transform (+3ms)
But its only like this for a few variable amount of records, usually 10, after that, the execution times are much worse:
Load Header(+972ms) Load Data (+990ms) Transform (+2ms)
Here you can see the complete function:
private void decodeWriteResponse(int identifier, object sender, SerialDataReceivedEventArgs e)
{
/* MEASURE TIME NOW */
long start = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
var serialPort = (SerialPort)sender; //new serial port object
int delay = ComPort.getWaitTime(7); //Returns the wait time (1s/baudrate * bytecount *8) +100ms Additional
Task.Delay(delay).Wait(); //wait until the device has send all its data!
byte[] databytes = new byte[6]; //new buffer
try
{
serialPort.Read(databytes, 0, 6); //read the data
}
catch (Exception) { };
/* MEASURE TIME NOW */
long between_header = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
/* Read the Data from Port */
int rec_len = databytes[1] | databytes[2] << 8; //Extract number of channels
int start_chnl = databytes[3] | databytes[4] << 8; //Extract the first channel
delay = ComPort.getWaitTime(rec_len+7); //get wait time
Task.Delay(delay).Wait(); //wait until the device has send all its data!
byte[] buf = new byte[rec_len-3]; //new buffer
try
{
serialPort.Read(buf, 0, rec_len-3); //read the data
}
catch (Exception) {}
/* MEASURE TIME NOW */
long after_load = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
/* Now perform spectrum analysis */
decodeSpectrumData(buf, start_chnl, rec_len-4);
/*MEASURE TIME NOW */
long end = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
Form.rtxtDataArea.AppendText("Load Header(+" + (between_header - start).ToString() + "ms) Load Data (+" + (after_load - between_header).ToString() + "ms) Transform (+" + (end - after_load) + "ms)\n");
/*Update the Write handler */
loadSpectrumHandler(1);
}
What could cause this issue?
I already tested this with "debug" in Visual Studio, and as "Release" standalone, but there is no difference.
Instead of trying to figure out how long a message will take to arrive at the port, why not just read the data in a loop until you have it all? For example, read the header and calculate the msg size. Then read that number of bytes. Ex:
// See if there are at least enough bytes for a header
if (serialPort.BytesToRead >= 6) {
byte[] databytes = new byte[6];
serialPort.Read(databytes, 0, 6);
// Parse the header - you have to create this function
int calculatedMsgSize = ValidateHeader(databytes);
byte [] msg = new byte[calculatedMsgSize];
int bytesRead = 0;
while (bytesRead < calculatedMsgSize) {
if (serialPort.BytesToRead) {
bytesRead += serialPort.Read(msg, bytesRead,
Math.min(calculatedMsgSize - bytesRead, serialPort.BytesToRead));
}
}
// You should now have a complete message
HandleMsg(msg);
}
I am working on a winodws IoT project that controls a led strip based on an audio input. Now do I have some code that gets the audio in and writes it to a buffer with the AudioGraph API, but I don't know how I can process the audio to some usefull data.
my code so far:
private async void MainPage_Loaded(object sender, RoutedEventArgs eventArgs)
{
try
{
// Initialize the led strip
//await this.pixelStrip.Begin();
sampleAggregator.FftCalculated += new EventHandler<FftEventArgs>(FftCalculated);
sampleAggregator.PerformFFT = true;
// Create graph
AudioGraphSettings settings = new AudioGraphSettings(AudioRenderCategory.Media);
settings.DesiredSamplesPerQuantum = fftLength;
settings.DesiredRenderDeviceAudioProcessing = Windows.Media.AudioProcessing.Default;
settings.QuantumSizeSelectionMode = QuantumSizeSelectionMode.ClosestToDesired;
CreateAudioGraphResult result = await AudioGraph.CreateAsync(settings);
if (result.Status != AudioGraphCreationStatus.Success)
{
// Cannot create graph
return;
}
graph = result.Graph;
// Create a device input node using the default audio input device
CreateAudioDeviceInputNodeResult deviceInputNodeResult = await graph.CreateDeviceInputNodeAsync(MediaCategory.Other);
if (deviceInputNodeResult.Status != AudioDeviceNodeCreationStatus.Success)
{
return;
}
deviceInputNode = deviceInputNodeResult.DeviceInputNode;
frameOutputNode = graph.CreateFrameOutputNode();
frameOutputNode.Start();
graph.QuantumProcessed += AudioGraph_QuantumProcessed;
// Because we are using lowest latency setting, we need to handle device disconnection errors
graph.UnrecoverableErrorOccurred += Graph_UnrecoverableErrorOccurred;
graph.Start();
}
catch (Exception e)
{
Debug.WriteLine(e.ToString());
}
}
private void AudioGraph_QuantumProcessed(AudioGraph sender, object args)
{
AudioFrame frame = frameOutputNode.GetFrame();
ProcessFrameOutput(frame);
}
unsafe private void ProcessFrameOutput(AudioFrame frame)
{
using (AudioBuffer buffer = frame.LockBuffer(AudioBufferAccessMode.Write))
using (IMemoryBufferReference reference = buffer.CreateReference())
{
byte* dataInBytes;
uint capacityInBytes;
float* dataInFloat;
// Get the buffer from the AudioFrame
((IMemoryBufferByteAccess)reference).GetBuffer(out dataInBytes, out capacityInBytes);
dataInFloat = (float*)dataInBytes;
}
}
So I end with my buffer as a float. But how can i change this to usefull data that makes it possible to create something like a spectrum analyzer?
Edit:
Maybe I have to make this question less specific for the audiograph. I use an API to get my audio input. The data I get from the API is a byte* and I can cast it to a float* How can I change it from the byte* or the float* to some other data that I can use to create some color codes.
I thaught about doing some FFT analysis on the float* to get 164 leds*3(rgb) = 492 bins. And process this data further to get some values between 0 and 255.
So how can I process this float* or byte* to get this usefull data? Or how do I start?
That data is interleaved IEEE float, so it alternates channel data as you step through the array, and the data range for each sample is from -1 to 1.
For example, a mono signal only has one channel, so it won't interleave data at all; but a stereo signal has two channels of audio, and so:
dataInFloat[0]
is the first sample of data from the left channel and
dataInFloat[1]
is the first sample of data from the right channel. Then,
dataInFloat[2]
is the second sample of data from the left channel. and they just keep going back and forth. All the other data you'll end up caring about is in windows.media.mediaproperties.audioencodingproperties
So, just knowing this, you (essentially) can immediately get the overall volume of the signal directly from this data by looking at the absolute value of each sample. You'll definitely want to average it out over some amount of time. You can even just attach EQ effects to different nodes, and make seperate Low, Mids, and High analyzer nodes and never even get into FFT stuff. BUT WHAT FUN IS THAT? (it's actually still fun)
And then, yeah, to get your complex harmonic data and make a truly sweet visualizer, you want to do an FFT on it. People enjoy using AForge for learning scenarios, like yours. See Sources/Imaging/ComplexImage.cs for usage, Sources/Math/FourierTransform.cs for implemenation
Then you can easily get your classic bin data and do the classic music visualizer stuff or get more creative or whatever! technology is awesome!
dataInFloat = (float*)dataInBytes;
float max = 0;
for (int i = 0; i < graph.SamplesPerQuantum; i++)
{
max = Math.Max(Math.Abs(dataInFloat[i]), max);
}
finalLevel = max;
Debug.WriteLine(max);
I'm trying to get the sound input and send output directly with less latency possible with C#.
I'm using the library NAudio that supports ASIO for better latency.
In particular, I use the AsioOut object for recording and another for playback initialized with a BufferedWaveProvider, which is filled in a Callback function: OnAudioAvailable, which allows me to use the ASIO buffers.
The problem is that I hear the sound with various glitches and with a bit of delay. I think the problem is in the function OnAudioAvailable where the buffer is filled with data taken as input from the sound card.
Declarations:
NAudio.Wave.AsioOut playAsio;
NAudio.Wave.AsioOut recAsio;
NAudio.Wave.BufferedWaveProvider buffer;
Play procedure:
if (sourceList.SelectedItems.Count == 0) return;
int deviceNumber = sourceList.SelectedItems[0].Index;
recAsio = new NAudio.Wave.AsioOut(deviceNumber);
recAsio.InitRecordAndPlayback(null, 2, 44100); //rec channel = 1
NAudio.Wave.WaveFormat formato = new NAudio.Wave.WaveFormat();
buffer = new NAudio.Wave.BufferedWaveProvider(formato);
recAsio.AudioAvailable += new EventHandler<NAudio.Wave.AsioAudioAvailableEventArgs>(OnAudioAvailable);
//Collego l'output col buffer
playAsio = new NAudio.Wave.AsioOut(deviceNumber);
playAsio.Init(buffer);
//Registro
recAsio.Play();
//Playback
playAsio.Play();
OnAudioAvailable():
//Callback
private unsafe void OnAudioAvailable(object sender, NAudio.Wave.AsioAudioAvailableEventArgs e)
{
//Copio tutti gli elementi di InputBuffers in buf e li aggiungo in coda al buffer
byte[] buf = new byte[e.SamplesPerBuffer];
for (int i = 0; i < e.InputBuffers.Length; i++)
{
Marshal.Copy(e.InputBuffers[i], buf, 0, e.SamplesPerBuffer);
//Aggiungo in coda al buffer
buffer.AddSamples(buf, 0, buf.Length);
}
}
AsioAudioAvailableEventArgs definition:
public AsioAudioAvailableEventArgs(IntPtr[] inputBuffers, int samplesPerBuffer, AsioSampleType asioSampleType);
public float[] GetAsInterleavedSamples();
Does anyone know how to fix it?
Thank you all.
You should not be using two instances of AsioOut for the same device. I'm surprised this works at all. Just use the one, with InitRecordAndPlayback.
For absolute minimal latency for pass-through monitoring, in AudioAvailableEvent, copy directly to the OutputBuffers, and set WrittenToOutputBuffers = true. This means you don't need a BufferedWaveProvider.
Also remember that any glitches are simply due to you not managing to process the AudioAvailable event quickly enough. When you're working with ASIO, you can be at very low latencies (e.g. sub 10ms), and can be dealing with lots of data (e.g. 96kHz sample rates, 8 channels of input and output). So you need to do a lot of moving of data in a short time window. With .NET, you have to factor in the unfortunate fact that the garbage collector could kick in at any time and cause you to miss a buffer from time to time.
I have been working on a private project where i wanted to learn how to program on a windows phone, and at a point i started to fiddle with sockets and the camera, and a great idea came to mind video feed (dumb me to even attempt).
but now I'm here, I have something that well, it works like a charm but a Lumia 800 cannot chug through the for-loop fast enough. It sends a frame per lets say 7-8 seconds something i think is strange since well, it should be strong enough. It feels and looks like watching porn on a 56k modem without the porn.
I also realized that a frame is 317000 pixels and that would sum up to roughly 1MB per frame I'm also sending xy coordinates so mine takes up 2.3MB per frame still working on a different way to solve this to keep it down. so I'm guessing i would need to do dome magic to make both position and pixel values of an acceptable size. because atm would i get5 it up at an acceptable speed it would require at least 60MB/s to get something like 30fps but thats a problem for another day.
//How many pixels to send per burst (1000 seems to be the best)
const int PixelPerSend = 1000;
int bSize = 7 * PixelPerSend;
//Comunication thread UDP feed
private void EthernetComUDP() //Runs in own thread
{
//Connect to Server
clientUDP = new SocketClientUDP();
int[] ImageContent = new int[(int)cam.PreviewResolution.Height * (int)cam.PreviewResolution.Width];
byte[] PacketContent = new byte[bSize];
string Pixel,l;
while (SendingData)
{
cam.GetPreviewBufferArgb32(ImageContent);
int x = 1, y = 1, SenderCount = 0;
//In dire need of a speedup
for (int a = 0; a < ImageContent.Length; a++) //this loop
{
Pixel = Convert.ToString(ImageContent[a], 2).PadLeft(32, '0');
//A - removed to conserve bandwidth
//PacketContent[SenderCount] = Convert.ToByte(Pixel.Substring(0, 8), 2);//0
//R
PacketContent[SenderCount] = Convert.ToByte(Pixel.Substring(8, 8), 2);//8
//G
PacketContent[SenderCount + 1] = Convert.ToByte(Pixel.Substring(16, 8), 2);//16
//B
PacketContent[SenderCount + 2] = Convert.ToByte(Pixel.Substring(24, 8), 2);//24
//Coordinates
//X
l = Convert.ToString(x, 2).PadLeft(16, '0');
//X bit(1-8)
PacketContent[SenderCount + 3] = Convert.ToByte(l.Substring(0, 8), 2);
//X bit(9-16)
PacketContent[SenderCount + 4] = Convert.ToByte(l.Substring(8, 8), 2);
//Y
l = Convert.ToString(y, 2).PadLeft(16, '0');
//Y bit(1-8)
PacketContent[SenderCount + 5] = Convert.ToByte(l.Substring(0, 8), 2);
//Y bit(9-16)
PacketContent[SenderCount + 6] = Convert.ToByte(l.Substring(8, 8), 2);
x++;
if (x == cam.PreviewResolution.Width)
{
y++;
x = 1;
}
SenderCount += 7;
if (SenderCount == bSize)
{
clientUDP.Send(ConnectToIP, PORT + 1, PacketContent);
SenderCount = 0;
}
}
}
//Close on finish
clientUDP.Close();
}
i have tried for simplicity to just send the pixels induvidialy using
BitConverter.GetBytes(ImageContent[a]);
instead of the string parsing mess i have created (to be fixed just wanted a proof of concept) but to do the simple BitConverter did not speed it up to much.
so now im on my last idea the UDP sender socket witch is rhoughly identical to the one on msdn's library.
public string Send(string serverName, int portNumber, byte[] payload)
{
string response = "Operation Timeout";
// We are re-using the _socket object that was initialized in the Connect method
if (_socket != null)
{
// Create SocketAsyncEventArgs context object
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs();
// Set properties on context object
socketEventArg.RemoteEndPoint = new DnsEndPoint(serverName, portNumber);
// Inline event handler for the Completed event.
// Note: This event handler was implemented inline in order to make this method self-contained.
socketEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(delegate(object s, SocketAsyncEventArgs e)
{
response = e.SocketError.ToString();
// Unblock the UI thread
_clientDone.Set();
});
socketEventArg.SetBuffer(payload, 0, payload.Length);
// Sets the state of the event to nonsignaled, causing threads to block
_clientDone.Reset();
// Make an asynchronous Send request over the socket
_socket.SendToAsync(socketEventArg);
// Block the UI thread for a maximum of TIMEOUT_MILLISECONDS milliseconds.
// If no response comes back within this time then proceed
_clientDone.WaitOne(TIMEOUT_MILLISECONDS);
}
else
{
response = "Socket is not initialized";
}
return response;
}
All in all i have ended up on 3 solutions
Accept defeat (but that wont happen so lets look at 2)
Work down the amount of data sent (destroys quality 640x480 is small enough i think)
Find the obvious problem (Google and friend's ran out of good ideas, thats why I'm here)
The problem is almost certainly the messing about with the data. Converting a megabyte of binary data into several megabytes of text and then extracting and sending individual characters will add a massive overhead per byte of source data. Looping through individual pixels to build a send buffer will take (relatively speaking) geological timescales.
The fastest way to do this is likely to be to grab a buffer of binary data from the camera, and send it with one UDP write. Only process or break up the data on the phone if you have to, and be careful to access the original binary data directly - don't convert it all to strings and back to binary. Every extra method call you add into this process will just add overhead. If you have to use a loop, try to pre-calculate as much as you can outside the loop to minimise the work that is done on each iteration.
A couple things come to mind: #1 Break up the raw image array into pieces to be sent over the wire. Not sure if Linq is available on Windows Phone but something like this. #2 Converting from int to string to byte will be very inefficient because of the processing time and memory usage. A better approach would be to bulk copy chunks of the int array to a byte array directly. Example
I try to stream sound samples from my microphone to my speakers by using DirectSound and C#. It should be similar to 'listening to microphone', but later I want to use this for something else. By testing my approach I've noticed silent tickeling, cracking noises in the background. I would guess this has something to do with the delay between writing and playing the buffer, which must be greater than the latency to write the chunks.
If I set the delay between recording and playout to less than 50ms. Mostly it works but sometimes I get really loud cracking noises. So I've decided to a delay about at least 50ms. This works okay for me, but the delay of the systems "listen to device" seems to be much shorter. I would guess it is about 15-30ms, and nearly not noticeable. For 50ms I get at least a little reverb effect.
In the following I'll show you my microphone code (partially):
The initialisation is done like this:
capture = new Capture(device);
// Creating the buffer
// Determining the buffer size
bufferSize = format.AverageBytesPerSecond * bufferLength / 1000;
while (bufferSize % format.BlockAlign != 0) bufferSize += 1;
chunkSize = Math.Max(bufferSize, 256);
bufferSize = chunkSize * BUFFER_CHUNKS;
this.bufferLength = chunkSize * 1000 / format.AverageBytesPerSecond; // Redetermining the buffer Length that will be used.
captureBufferDescription = new CaptureBufferDescription();
captureBufferDescription.BufferBytes = bufferSize;
captureBufferDescription.Format = format;
captureBuffer = new CaptureBuffer(captureBufferDescription, capture);
// Creating Buffer control
bufferARE = new AutoResetEvent(false);
// Adding notifier to buffer.
bufferNotify = new Notify(captureBuffer);
BufferPositionNotify[] bpns = new BufferPositionNotify[BUFFER_CHUNKS];
for(int i = 0 ; i < BUFFER_CHUNKS ; i ++) bpns[i] = new BufferPositionNotify() { Offset = chunkSize * (i+1) - 1, EventNotifyHandle = bufferARE.SafeWaitHandle.DangerousGetHandle() };
bufferNotify.SetNotificationPositions(bpns);
The capturing will run like this in an extra thread:
// Initializing
MemoryStream tempBuffer = new MemoryStream();
// Capturing
while (isCapturing && captureBuffer.Capturing)
{
bufferARE.WaitOne();
if (isCapturing && captureBuffer.Capturing)
{
captureBuffer.Read(currentBufferPart * chunkSize, tempBuffer, chunkSize, LockFlag.None);
ReportChunk(applyVolume(tempBuffer.GetBuffer()));
currentBufferPart = (currentBufferPart + 1) % BUFFER_CHUNKS;
tempBuffer.Dispose();
tempBuffer = new MemoryStream(); // Reset Buffer;
}
}
// Finalizing
isCapturing = false;
tempBuffer.Dispose();
captureBuffer.Stop();
if (bufferARE.WaitOne(bufferLength + 1)) currentBufferPart = (currentBufferPart + 1) % BUFFER_CHUNKS; // That on next start the correct bufferpart will be read.
stateControlARE.Set();
While capturing ReportChunk takes the data to the speaker as an event that could be subscribed. The speaker part is initialized like this:
// Creating the dxdevice.
dxdevice = new Device(device);
dxdevice.SetCooperativeLevel(hWnd, CooperativeLevel.Normal);
// Creating the buffer
bufferDescription = new BufferDescription();
bufferDescription.BufferBytes = bufferSize;
bufferDescription.Format = input.Format;
bufferDescription.ControlVolume = true;
bufferDescription.GlobalFocus = true; // That sound doesn't stop if the hWnd looses focus.
bufferDescription.StickyFocus = true; // - " -
buffer = new SecondaryBuffer(bufferDescription, dxdevice);
chunkQueue = new Queue<byte[]>();
// Creating buffer control
bufferARE = new AutoResetEvent(false);
// Register at input device
input.ChunkCaptured += new AInput.ReportBuffer(input_ChunkCaptured);
The data is put by the event method into the queue, simply by:
chunkQueue.Enqueue(buffer);
bufferARE.Set();
Filling the playbackbuffer and starting/stopping the playback buffer is done by another thread:
// Initializing
int wp = 0;
bufferARE.WaitOne(); // wait for first chunk
// Playing / writing data to play buffer.
while (isPlaying)
{
Thread.Sleep(1);
bufferARE.WaitOne(BufferLength * 3); // If a chunk is played and there is no new chunk we try to continue and may stop playing, else may the buffer runs out.
// Note that this may fails if the sender was interrupted within one chunk
if (isPlaying)
{
if (chunkQueue.Count > 0)
{
while (chunkQueue.Count > 0) wp = writeToBuffer(chunkQueue.Dequeue(), wp);
if (buffer.PlayPosition > wp - chunkSize * 3 / 2) buffer.SetCurrentPosition(((wp - chunkSize * 2 + bufferSize) % bufferSize));
if (!buffer.Status.Playing)
{
buffer.SetCurrentPosition(((wp - chunkSize * 2 + bufferSize) % bufferSize)); // We have 2 chunks buffered so we step back 2 chunks and play them while getting new chunks.
buffer.Play(0, BufferPlayFlags.Looping);
}
}
else
{
buffer.Stop();
bufferARE.WaitOne(); // wait for a filling chunk
}
}
}
// Finalizing
isPlaying = false;
buffer.Stop();
stateControlARE.Set();
writeToBuffer simply writes the enqueued chunk to the buffer by this.buffer.Write(wp, data, LockFlag.None); and caring about bufferSize and chunkSize and wp, which represents the last writing position. I think this is everything that is important about my code. Maybe the definitions are missing and at least there is another method that starts/stops=controls the threads.
I've posted this code in case I've made a mistake in filling the buffer or my initialisation is wrong. But I would guess that this problem occurs because the execution of C# bytecode is too slow or something like that. But in the end my question is still open: My question is how to reduce the latency and how to avoid noises that shouldn't be there?
I know the reason of your problem and the way that you can solve it, but I can't implement it in C# and .Net, so I will explain it in hope that you can find your way.
Audio will be recorded by your mic. with an specified frequency( for example 44100 ) and then played on the sound card at same sample rate( again 44100 ), the problem is the crystal that count the time in input device( mic. for example ) is not same as the crystal that play sound in sound card.
also the difference is so small they are not the same( there is no 2 exact same crystal in entire world ) so after a while there will be a gap in your playback routines.
Now the solution is re-sample data to match the sample rate of the output but I don't know how to do that in C# and .Net
A long time ago I figured out, that this problem was caused by the Thread.Sleep(1); in combination with high CPU usage. Due the windows timerresolution is 15,6ms by default, this sleep doesn't mean sleep for 1ms, but sleep until the next clock interrupt is reached. (For more read this paper) Combined with high CPU usage it may stacks up to the length of a chunk or even more.
For example: If my chunksize is 40ms, this could be about 46,8ms (3 * 15,6ms) and this causes the tickeling. One solution for that is setting the resolution down to 1ms. That can be done in this way:
[DllImport("winmm.dll", EntryPoint="timeBeginPeriod", SetLastError=true)]
private static extern uint timeBeginPeriod(uint uiPeriod);
[DllImport("winmm.dll", EntryPoint="timeEndPeriod", SetLastError=true)]
private static extern uint timeEndPeriod(uint uiPeriod);
void routine()
{
Thead.Sleep(1); // May takes about 15,6ms or even longer.
timeBeginPeriod(1); // Should be set at the startup of the application.
Thead.Sleep(1); // May takes about 1, 2 or 3 ms depending on the CPU usage.
// ... time depending routines goes here ...
timeEndPeriod(1); // Should end at application shutdown.
}
As far as I know this should be already done by directx. But due this setting is a global setting, other parts of the application or other applications maybe change it. This shouldn't happen if an application sets and revokes the setting once. But somehow it seems to happen caused by any dirty programmed part or other running application.
One more thing that needs to be watched, is whether you're still using the correct position of the directx buffer, if you're skipping one chunk for any reason. In this case a resynchronization is required.