I would like transfer mixed sound from WCF server to all connected clients. Using WCF service callbacks for this. Sound is mixed using naudio library.
Here is little example of server-side (WCF method):
MixingSampleProvider _mixer = new MixingSampleProvider(sound32.WaveFormat);
SampleToWaveProvider _sampleToWave = new SampleToWaveProvider(_mixer);
// service method
byte[] buffer = new byte[1000];
do{
_sampleToWave.Read(buffer, 0, 1000);
client.Callback.SendBuffer(buffer);
} while (_isPlaying)
and client-side:
BufferedWaveProvider _bufferedWave = new BufferedWaveProvider(sound32.WaveFormat);
// DirectSoundOut _output = new DirectSoundOut();
WaveOut _output = new WaveOut();
_output.Init(_bufferedWave);
// callback event method
if (_output.PlaybackState != PlaybackState.Playing)
_bufferedWave.AddSamples(buffer, 0, 1000);
// now in timer_tick event method
// if(_bufferedWave.BufferedDuration.TotalSeconds > 0.5)
// _output.Play();
// else
// _output.Pause();
I'm new in this, so I have a few questions.
Is this idea a good one? Is there simpler option to handle this?
[EDIT_1] I created test app with local two methods, which should simulate this and I found, that _bufferedWave.BufferedBytes are not clearing when is buffered sound playing (and it will overflow immediately). Can somebody tell me, why?
[EDIT_1] Changed type of _output field from DirectSoundOut to WaveOut and it's helpful.
Second change I did was, that I added DispatcherTimer to handle when is buffered duration greater than 0.5 (according naudio MP3Streaming example).
Now, I'm fighting with buffer time. I can hear sound only for time in _timer_Tick event method:
_bufferedWave.BufferedDuration.TotalSeconds > XX // this XX is time I can hear sound
Any ideas or opinions?
I'm not sure that this will work the way you hope. WCF is TCP based and TCP is not designed for broadcasting audio video or anything that requires speed (like games) due to it's constant checking of packets.
I have previously used naudio to transmit sound over a network to a listener client but for it to work you will need to use UDP.
If you want this to work over the internet then you will need to look into UDP hole-punching.
Also before you transmit your audio you should compress it from 16bit to 8bit and then back to 16bit upon receiving it using something like an ALaw or MuLaw Decoder/Encoder.
Related
dso = new DirectSoundOut(Guid.Parse(AudioOutDevice));
var ms = new MemoryStream(soundArray.ToArray()))
{
IWaveProvider provider = new RawSourceWaveStream(ms, new WaveFormat());
dso.Init(provider);
dso.Play();
Thread.Sleep(3000);
}
I am able to play sound array through desired output device using the above code, and i am unable to hear the sound if there is thread.sleep. But I am unable to understand the reason for using thread.sleep. Can any one let me know the reason for thread.sleep()
The call to Play is not blocking. It simply starts playback. So you must keep dso alive until playback ends or you have stopped it manually.
You can use code like this if you want to block yourself (obviously only use this if your audio isn't infinitely long)
dso.Play();
while (dso.PlaybackState == PlaybackState.Playing)
{
Thread.Sleep(500);
}
dso.Dispose();
I've written a simple .Net client that connects to a hardware device (FPGA) over TCP/IP. When I click a button in the client it sends a small (4 byte) "request" to the device, then immediately reads a block of data (approx 32kb) that the device responds with. The client code looks something like this:-
var stream = _tcpClient.GetStream();
stream.Write(requestData, 0, requestData.Length);
using (var ms = new MemoryStream())
{
var tempBuffer = new byte[65535];
do
{
var numBytesRead = stream.Read(tempBuffer, 0, tempBuffer.Length);
ms.Write(tempRead, 0, numBytesRead);
}
while(ms.Length < ExpectedResponseSize);
_hardwareResponse = ms.ToArray();
}
Using a Stopwatch in the above code typically reports 2-3ms to read the entire 32kb response back, and this timing remains consistent if I repeatedly click the button slowly (e.g. once per second).
If I start clicking the button more rapidly (e.g. every half a second) then after a few seconds the timings suddenly drop to around 12ms and stay there, even if I go back to clicking the button slowly. If I close then reopen the connection on the client and try again, it's back to 2-3ms times.
WireShark shows 3-4 ACKs coming out of the PC during the faster responses, but this increases to a dozen or more once the timing drops to 12ms. In both cases the number and size of packets coming from the FPGA is the same. I'm as confident as I can be that it's not a problem in the code on the client or FPGA (neither can get much simpler) - gut feeling is that it's a protocol or network thing. Any thoughts?
Have you looked at the various answers in this similar question: https://serverfault.com/questions/215674/latency-in-tcp-ip-over-ethernet-networks/215963 ? I suspect that the issue is window size or ack algorithm reconfiguring itself pessimistically, but the only real way to get to the bottom is to try adjusting one thing at a time on the client and server until you spot the performance difference. A lot of adaptive algorithms (like Nagle) are in use by default and can sometimes have unintended consequences on local traffic.
I'm using WaveInEvent of NAudio to record microphone data. It works fine for a while, but after a few times, it stops providing input data- the DataAvailable callback is never called with new data.
I have tried creating a new WaveInEvent each time, but this did not resolve the problem. I've also tried using the WASAPI input, which always called DataAvailable - with zero bytes of data.
How can I record audio from the microphone reliably with NAudio?
Currently, my code looks like this:
StartRecording() {
microphone = new WaveInEvent();
microphone.DeviceNumber = 0;
microphone.WaveFormat = outformat;
microphone.BufferMilliseconds = 50;
microphone.DataAvailable += (_, recArgs) =>
{
session.OnAudioData(recArgs.Buffer, recArgs.BytesRecorded);
};
microphone.StartRecording();
}
StopRecording() {
if (microphone != null)
{
microphone.StopRecording();
microphone.Dispose();
microphone = null;
}
}
There's no other naudio code in the project except using WaveFormat to describe wave formats.
NAudio throws an access violation exception trying to call WaveInBuffer.Reuse() from a threadpool worker. I'm not sure why this doesn't do something more serious than just drop audio data.
For the condition where I did not recreate the WaveInEvent, I get an MmException instead- invalid handle calling waveInPrepareHeader, in the same place.
Frankly, the fact that I get different results heavily implies that NAudio is doing some funky shit it shouldn't to share state between instances, and looking at the source on Codeplex, I'm not really sure WTF is going on.
It seems that the drivers for the USB microphone do not behave correctly. When the buffer is sent to the user through the WIM_DATA message, it is full. However when waveInUnprepareHeader is called, it's still in the queue, even though it was literally just passed as full. So I think that the drivers for the microphone are ultimately to blame.
I've been looking more closely at the microphone and it seems that this particular unit is actually known to have been damaged.
I have run into an issue with the slow C# start-up time causing UDP packets to drop initially. Below, I is what I have done to mitigate this start-up delay. I essentially wait an additional 10ms between the first two packet transmissions. This fixes the initial drops at least on my machine. My concern is that a slower machine may need a longer delay than this.
private void FlushPacketsToNetwork()
{
MemoryStream packetStream = new MemoryStream();
while (packetQ.Count != 0)
{
byte[] packetBytes = packetQ.Dequeue().ToArray();
packetStream.Write(packetBytes, 0, packetBytes.Length);
}
byte[] txArray = packetStream.ToArray();
udpSocket.Send(txArray);
txCount++;
ExecuteStartupDelay();
}
// socket takes too long to transmit unless I give it some time to "warm up"
private void ExecuteStartupDelay()
{
if (txCount < 3)
{
timer.SpinWait(10e-3);
}
}
So, I am wondering is there a better approach to let C# fully load all of its dependencies? I really don't mind if it takes several seconds to completely load; I just do not want to do any high bandwidth transmissions until C# is ready for full speed.
Additional relevant details
This is a console application, the network transmission is run from a separate thread, and the main thread just waits for a key press to terminate the network transmitter.
In the Program.Main method I have tried to get the most performance from my application by using the highest priorities reasonable:
public static void Main(string[] args)
{
Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High;
...
Thread workerThread = new Thread(new ThreadStart(worker.Run));
workerThread.Priority = ThreadPriority.Highest;
workerThread.Start();
...
Console.WriteLine("Press any key to stop the stream...");
WaitForKeyPress();
worker.RequestStop = true;
workerThread.Join();
Also, the socket settings I am currently using are shown below:
udpSocket = new Socket(targetEndPoint.Address.AddressFamily,
SocketType.Dgram,
ProtocolType.Udp);
udpSocket.Ttl = ttl;
udpSocket.SendBufferSize = 1024 * 1024;
udpSocket.Blocking = true;
udpSocket.Connect(targetEndPoint);
The default SendBufferSize is 8192, so I went ahead and moved it up to a megabyte, but this setting did not seem to have any affect on the dropped packets at the beginning.
From the comments I learned that TCP is not an option for you (because of inherent delays in transmission), also you do not want to loose packets due to other side being not fully loaded.
So you actually need to implement some features present in TCP (retransmission) but in more robust and lightweight fashion. I also assume that you are in control of the receiving side.
I propose that you send some predetermined number of packets. And then wait for confirmation. For instance, every packet can have an id that constantly grows. Every N packets, receiving application sends the number of last received packet to the sender. After receiving this number sender will know if it is necessary to repeat last N packets.
This approach should not hurt your bandwidth very much and you will get some sort of information about received data (although not guaranteed).
Otherwise it is best to switch to TCP. By the way did you try using TCP? How much your bandwidth hurts because of it?
I am playing with using TTS built into .NET 4 and want the speech to happen immediately, but am instead encountering a lag between when I call Speak and when I get the audio.
I am developing a simple count-down timer that calls off the last five seconds and completion (5... 4... 3... 2... 1... Done), but when the screen updates with the new time, the TTS lags behind, getting worse for every invocation. I tried using SpeakAsync, but this only made it worse. Currently, Speak is being called outside the UI thread (in the Timer Tick event handler).
Is there a way to minimize this lag, such as pre-computing the speech and caching it or creating some kind of special TTS thread?
I somehow read past the API call I needed at least a hundred times. I was looking for SpeechSynthesizer.SetOutputToWaveStream.
MemoryStream stream = new MemoryStream();
SpeechSynthesizer synth = new SpeechSynthesizer();
synth.SetOutputToWaveStream(stream);
synth.Play(text);
stream.Position = 0;
SoundPlayer player = new SoundPlayer(stream);
player.Play();
This code will use TTS to turn text into a WAV file that is streamed into stream. You need to reset the position of the MemoryStream so that when you create a SoundPlayer from it, it starts reading the stream from the beginning instead of the end. Once you have the SoundPlayer initialized, you can save it somewhere so you can play it later instantly instead of having to wait for the TTS to initialize and play the sound.