We Keep Coding

(c#) Playing and controlling multiple sound files using DirectShow

I have very little experience with DirectShow, so far I have managed paly a .wav file over a particular output device while being able to control its volume and get/set its track-position. Basically I’m able to create a very simple sound player application.
Here is the code I’m currently using:
//select an output device
DsDevice[] devices = DsDevice.GetDevicesOfCat(FilterCategory.AudioRendererCategory
DsDevice device = (DsDevice)devices[xxx];
//define a source
Guid iid = typeof(IBaseFilter).GUID;
object source = null;
device.Mon.BindToObject(null, null, ref iid, out source);
//build a basic graph to play the sound
IGraphBuilder player_gra = (IGraphBuilder)new FilterGraph();
player_gra.AddFilter((IBaseFilter)source, "Audio Render");
player_gra.RenderFile(#"test.wav", "");
//start the sound
(player_gra as IMediaControl).Run();
//control the volume
(player_gra as IBasicAudio).put_Volume(volume);
//get/set position on the track
(player_gra as IMediaPosition).get_Duration(out out1);//how long the track is
(player_gra as IMediaPosition).get_CurrentPosition(out out2);
(player_gra as IMediaPosition).put_CurrentPosition(yyy);
What I would like to do now is to play several .wav files simultaneously while being able to control the volume and track-position of each file at runtime.
My first attempt was to create several IGraphBuilder instances and run them at the same time but it seem that only one can play at the same time while the others wait until the currently playing one is terminated via:
Marshal.ReleaseComObject(player_gra);
My second attempt was to give the IGraphBuilder several Files to render before starting it.
…
player_gra.RenderFile(#"testA.wav", "");
player_gra.RenderFile(#"testB.wav", "");
player_gra.RenderFile(#"testC.wav", "");
…
This way the files are played simultaneously but I see no way to control the volume of each individual sound, much less its position on the audio track.
Thank you in advance ;-)

In these lines
DsDevice[] devices = DsDevice.GetDevicesOfCat(FilterCategory.AudioRendererCategory);
DsDevice device = (DsDevice)devices[0];
you enumerate audio output devices and pick the first "random" device which appears to be Default WaveOut Device or one of its instances.
It has a legacy behavior that only one active instance is actually sending data to playback. There is no fundamental limitation in the system to prevent from simultaneous plyback, it is just legacy behavior.
That is, you have both graph playing but audio from the second is muted.
Audio mixing is disabled in the waveOut Audio Renderer, so if you need to mix multiple audio streams during playback, use the DirectSound renderer.
If you use Default DirectSound Device instead (which you can quickly pick up by using different index in device[NNN] in code) you'll hear what you expect to hear.
DirectShow.NET does the enumeration somehow confusingly, Default DirectSound Device normally has highest merit and is listed first and you seem to be given devices in different order.

Is it possible to supply raw audio (byte stream) data in UWP application?

I have a custom Audio DAC device. It has own controller inside and knows how to work with a certain byte stream. There is a library I'm trying to use:
Audio Graph. I generate some data in memory and send it like so:
private unsafe AudioFrame GenerateAudioData(byte[] data)
{
uint bufferSize = 7000000;
AudioFrame frame = new Windows.Media.AudioFrame(bufferSize);
using (AudioBuffer buffer = frame.LockBuffer(AudioBufferAccessMode.Write))
using (IMemoryBufferReference reference = buffer.CreateReference())
{
byte* dataInBytes;
uint capacityInBytes;
((IMemoryBufferByteAccess)reference).GetBuffer(out dataInBytes, out capacityInBytes);
for (int i = 0; i < bufferSize; i += 4)
{
dataInBytes[i] = data[i];
dataInBytes[i + 1] = data[i + 1];
dataInBytes[i + 2] = data[i + 2];
dataInBytes[i + 3] = data[i + 3];
}
}
return frame;
}
Here my audio graph settings:
AudioGraphSettings settings = new AudioGraphSettings(AudioRenderCategory.Other)
{
EncodingProperties = AudioEncodingProperties.CreatePcm(44100, 2, 16),
AudioRenderCategory = AudioRenderCategory.Other,
DesiredRenderDeviceAudioProcessing = AudioProcessing.Raw
};
The problem is that something modify my stream and physical device doesn't recieve exactly the same data. On sound it is not noticeable, but I need to deliver the same bytes to the endpoint device. Using WASAPI I don't have such problem. Also it would be better if I get an exclusive access to my device. It is highly undesirable that the system sounds/alerts/notifications are mixed with my audio stream.
Thanks in advance!

I notice that your audio data use a 44.1 kHz sampling rate. Many audio devices support 48 kHz and this is the sampling rate used as the device's "mix format". In that case, AudioGraph will resample your audio data to match the "mix format" so that your data can be mixed with system sounds and sounds from other apps.
AudioGraph cannot send the data as-is to the audio device because that would require opening the audio device in exclusive mode, which would block any system sounds, would interfere with Cortana, etc. AudioGraph does not support opening the audio device in exclusive mode.
As a possible work-around, if you could force your audio device to support 44.1 kHz only, then this would become the new "mix format" and AudioGraph would not resample the audio. Some audio devices support this through the property page that controls low-level audio settings.
Another work-around is to use the IAudioClient API (WASAPI) to open the audio device in exclusive mode.

Microsoft SpeechSynthesizer crackles when outputting to files and streams

I'm writing a thing that uses the SpeechSynthesizer to generate wave files on request, but I'm having problems with crackling noises. The weird thing is that output directly to the sound card is just fine.
This short powershell script demonstrates the issue, though I'm writing my program in C#.
Add-Type -AssemblyName System.Speech
$speech = New-Object System.Speech.Synthesis.SpeechSynthesizer
$speech.Speak('Guybrush Threepwood, mighty pirate!')
$speech.SetOutputToWaveFile("${PSScriptRoot}\foo.wav")
$speech.Speak('Guybrush Threepwood, mighty pirate!')
What this should do, is output to the speakers, and then save that same sound as "foo.wav" next to the script.
What it does is output to the speakers, and then save a crackling, old record player sounding version as a wave file. I've tested this on three different machines, and though they select different voices by default (all Microsoft provided default ones), they all sound like garbage falling down stairs in the wave file.
Why?
EDIT: I am testing this on Windows 10 Pro, with the latest updates that add that annoying "People" button on the taskbar.
EDIT 2: Here's a link to an example sound generated with the above script. Notice the crackling voice, that's not there when the script outputs directly to the speakers.
EDIT 3: It's even more noticeable with a female voice
EDIT 4: The same voice as above, saved to file with TextAloud 3 - no cracking, no vertical spikes.

I find it hard to believe this is a PoSH issue.
It's not PoSH doing the encoding on the serialization to disk. Its the API/Class that is being used.
'msdn.microsoft.com/en-us/library/system.speech.synthesis.speechsynthesizer(v=vs.110).aspx'
As per the MSDN, there is no option to control the encoding, bit rate, etc.
.wav has never been HQ stuff. So, I'd wonder if you take that .wav through a converter to make it an .mp3 or mp4, if that would correct your quality concerns. But that also means getting the converter on users systems.
Secondly, since Win8, the default player does not even play .wav correctly or at all. Sure, you can still set the default play of .wav to Windows Media Player or call the file via VLC, but it's still a .wav file. Yet, that also means, you having to set the Media Player assignment on every target system.

This is an issue with the SpeechSynthesizer API, which simply provides bad quality, crackling audio as seen in the samples above. The solution is to do what TextAloud does, which is to use the SpeechLib COM objects directly.
This is done by adding a COM reference to "Microsoft Speech Object Library (5.4)". Here is a snippet of the code I ended up with, which produces audio clips of the same quality as TextAloud:
public new static byte[] GetSound(Order o)
{
const SpeechVoiceSpeakFlags speechFlags = SpeechVoiceSpeakFlags.SVSFlagsAsync;
var synth = new SpVoice();
var wave = new SpMemoryStream();
var voices = synth.GetVoices();
try
{
// synth setup
synth.Volume = Math.Max(1, Math.Min(100, o.Volume ?? 100));
synth.Rate = Math.Max(-10, Math.Min(10, o.Rate ?? 0));
foreach (SpObjectToken voice in voices)
{
if (voice.GetAttribute("Name") == o.Voice.Name)
{
synth.Voice = voice;
}
}
wave.Format.Type = SpeechAudioFormatType.SAFT22kHz16BitMono;
synth.AudioOutputStream = wave;
synth.Speak(o.Text, speechFlags);
synth.WaitUntilDone(Timeout.Infinite);
var waveFormat = new WaveFormat(22050, 16, 1);
using (var ms = new MemoryStream((byte[])wave.GetData()))
using (var reader = new RawSourceWaveStream(ms, waveFormat))
using (var outStream = new MemoryStream())
using (var writer = new WaveFileWriter(outStream, waveFormat))
{
reader.CopyTo(writer);
return o.Mp3 ? ConvertToMp3(outStream) : outStream.GetBuffer();
}
}
finally
{
Marshal.ReleaseComObject(voices);
Marshal.ReleaseComObject(wave);
Marshal.ReleaseComObject(synth);
}
}
This is the code to convert a wave file to mp3. It uses NAudio.Lame from nuget.
internal static byte[] ConvertToMp3(Stream wave)
{
wave.Position = 0;
using (var mp3 = new MemoryStream())
using (var reader = new WaveFileReader(wave))
using (var writer = new LameMP3FileWriter(mp3, reader.WaveFormat, 128))
{
reader.CopyTo(writer);
mp3.Position = 0;
return mp3.ToArray();
}
}

Streaming video (C# using FFmpeg AutoGen) sends multiple data requests

I've written a video generator that rights a video in h264 format (mp4). When I stream the video from my azure service, i'm seeing the following network traffic:
The AVCodecContext layout I'm using is as follows:
AVCodec* videoCodec = ffmpeg.avcodec_find_encoder(AVCodecID.AV_CODEC_ID_H264)
AVCodecContext* videoCodecContext = ffmpeg.avcodec_alloc_context3(videoCodec);
videoCodecContext->bit_rate = 400000;
videoCodecContext->width = 1280;
videoCodecContext->height = 720;
videoCodecContext->gop_size = 12;
videoCodecContext->max_b_frames = 1;
videoCodecContext->pix_fmt = videoCodec->pix_fmts[0];
videoCodecContext->codec_id = videoCodec->id;
videoCodecContext->codec_type = videoCodec->type;
videoCodecContext->time_base = new AVRational
{
num = 1,
den = 30
};
ffmpeg.av_opt_set(videoCodecContext->priv_data, "preset", "ultrafast");
I'm also tried setting the "movflags" option for avformat_write_header() via an AVDictionary, but then av_write_trailer() returns -2, cause the file to not finish writing.
I cannot figure out how to solve this problem. Videos generating using Windows Movie Maker stream perfectly.
I know this has something to do with mdat and mov positions.
Also, this appears to only happening in Google Chrome.

OK, figured this out. I've been writing the video frames first and the audio frames afterwards. Instead, you have to write them side by side in order for faststart to actually work and allow the video to stream.
So, write a specific amount of audio and then determine if a video frame should be written by checking the timebases against the current writing indexes.
This example will show you how its done.
Also, to get the video and audio streams to have accurate PTS/DTS values, look at this question.

Playing Audio File using NAudio 1.7+

I was having issues playing back network streams as well as audio files using AAC and MP3 using NAudio. Took a while to figure out but the below solution works.

Hope to helps anyone else that is having a hard time utilizing NAudio properly.
//Create Output Stream with Data from Audio File / Network Stream
WaveOutputStream outputStream = new MediaFoundationReader("Path to File");
//Create Volume Stream to control volume of output
//ex: volumeStream.Volume = 0.5f; Float between 0 & 1
WaveChannel32 volumeStream = new WaveChannel32(outputStream);
//Create WaveOutEvent since it works in Background and UI Threads
WaveOutEvent player = new WaveOutEvent();
//Init Player with Configured Volume Stream
player.Init(volumeStream);
player.Play();
This code can play any audio file that MediaFoundationReader supports (MP3,AAC,WAV) as well as network streams of these codecs. To reuse the above player call Dispose() on player,outputStream, and volumeStream then set each to null.