I'm looking to develop a Silverlight application which will take a stream of data (not an audio stream as such) from a web server.
The data stream would then be manipulated to give audio of a certain format (G.711 a-Law for example) which would then be converted into PCM so that additional effects can be applied (such as boosting the volume).
I'm OK up to this point. I've got my data, converted the G.711 into PCM but my problem is being able to output this PCM audio to the sound card.
I basing a solution on some C# code intended for a .Net application but in Silverlight there is a problem with trying to take a copy of a delegate (function pointer) which will be the topic of a separate question once I've produced a simple code sample.
So, the question is... How can I output the PCM audio that I have held in a data structure (currently an array) in my Silverlight to the user? (Please don't say write the byte values to a text box)
If it were a MP3 or WMA file I would play it using a MediaElement but I don't want to have to make it into a file as this would put a crimp on applying dynamic effects to the audio.
I've seen a few posts from people saying low level audio support is poor/non-existant in Silverlight so I'm open to any suggestions/ideas people may have.
The simple answer is that there is no support for PCM playback from Silverlight in version 2. So unless you want to write a fully managed PCM to MP3 converter you are stuck. Even then I'm not sure you could get the MediaElement to play from isolated storage.
Is there any chance you could use a web service to perform the conversion?
See also this question:
Where's the sound API in Silverlight? Or, how do I write a music app to run in the browser?
Update: Silverlight 3 supports your custom audio sources. However, it won't let you intercept samples to perform effects on WMA or MP3, presumably for DRM reasons, so you would still potentially need to write your own decoder.
Short answer is use a MediaElement + a MediaStreamSource
Check out these:
http://blogs.msdn.com/gillesk/archive/2009/03/23/playing-back-wave-files-in-silverlight.aspx
http://code.msdn.microsoft.com/wavmss/Release/ProjectReleases.aspx?ReleaseId=2417
Basically, write a decoder in managed code to convert G.711 a-Law to PCM, then do whatever modifications you want to the raw values, then pass those into a MediaStreamSource.
Looks like Silverlight 3 supports direct PCM output now, or will when released. I don't see anything in the docs about the raw AV pipeline yet.
Mark Heath's answer is correct - only certain formats are supported - mp3 and certain flavours of WMA (unfortunately not WMA lossless which would be 'closer' to PCM).
To play PCM data in Silverlight, you could do the following:
* Convert the PCM into mp3 data, and store it in memory.
* Play the mp3 data using the technique presented at ManagedMediaHelpers. The idea here involves a class called Mp3MediaStreamSource (derived from System.Windows.Media.MediaStreamSource) that provides mp3 chunks to a MediaElement to play. The chunks will need to be in a stream, but of course a memory stream will do.
I initially thought you might be able to provide PCM chunks via MediaStreamSource, but this does not work. It's a real shame as it would solve your problem (and the one I was facing - making a Speex audio file player) really easily!
Related
Thank you for your answer, I really appreciate it, and I have to share with you that it was a useful example, but in the following link is explained how to use it on Windows Phone 8.1 and how to create your own implementation of MSS.
http://video.ch9.ms/sessions/build/2014/2-528.pptx
What you’re doing is impossible.
Wave files aren’t media streams. You can’t dynamically change a .wav file and expect the MediaElement to pick those changes.
If you’re trying to play audio that you’re receiving from the network, or audio that you're generating dynamically from something else, then you need to play from your custom media stream source, not from a file.
The sample code is available here. The sample plays mp3; you’ll need to change it, wave audio is supported, with some limitations.
I’m making an audio synthesizer and I’m having issues figuring out what to use for audio playback. I’m using physics and math to calculate the source waveforms and then need to feed that waveform to something which can play it as sound. I need something that can 1) play the waveforms I calculate and 2) play multiple sounds simultaneously (like holding one key down on a piano while pressing other keys). I’ve done a fair bit of research into this and I can’t find something that does both of those things. As far as I know, I have 5 potential options:
DirectSound. It can take a waveform (a short[]) as a parameter and play it as sound, and can play multiple sounds simultaneously. But it won’t work with .NET 4.5.
System.Media.SoundPlayer. It works with .NET 4.5 and has better quality audio than Direct Sound, but it has to play sound from a .wav file and cannot play multiple sounds at once (nor can multiple instances of SoundPlayer). I ‘trick’ SoundPlayer into working by translating my waveform into .wav format in memory and then send SoundPlayer a MemoryStream of the in-memory .wav file. Could I potentially achieve control over the playback by altering the stream? I cannot append bytes to the stream (I tried) but I could potentially make the stream an arbitrary size and just re-write all the bytes in the stream with the next segment of audio data every time the end of the stream is reached.
System.Windows.Controls.MediaElement. I have not experimented with this yet, but from MSDNs documentation I don’t see a way to send it a waveform in memory without saving it to disk first and then reading it; I don’t think I can send it a stream.
System.Windows.Controls.MediaPlayer. I have not experimented with this either, but the documentation says it’s meant to be used as a companion to some kind of animation. I could potentially use this without doing any real (user-perceivable) animation to achieve my desired effect.
An open source solution. I’m hesitant to use an open source solution as I find they are typically poorly documented and not very maintainable, but I am open to ideas if there is one out there that is well documented and can do what I need.
Can anyone offer me any guidance on this or how to create flexible audio playback?
http://naudio.codeplex.com , without a doubt. Mark is a regular here on SO, the product is well alive, there are good code examples.
It works. We built some great stuff with it.
I am developing a windows phone 7 application and it does video recording. I would like to get the sound portion of the video file (MP4) and do some enhancements on the sound. I believe sound is saved as AAC frames in MP4. (Right?) How can I extract sound of a videa MP4 file?
Since this is a video file, it can be huge file. So uploading to cloud and processing there is not a good option. Since this WP7 application I cannot use unmaged dlls :( Is there a way to do in pure C#? Any open source tools/samples?
Thanks!
MP4 is a container format and realistically the sound portion isn't always AAC. It could be MP3 or any other number of different audio formats. You may be thinking of M4A, which I believe requires either AAC or ALAC.
On the subject of audio extraction, it should be possible to extract the audio from an MP4 using just managed code. You'll have to read up on the MP4 format (here, for example - this question is also worth reading) and then search through the file for the location of the audio and then either copy it to its own buffer or do your manipulations in chunks. Even then, you'll have to be able to recognize when it isn't an audio format that your app won't support.
It's possible that there already exists a .net library that can do all of this but I don't know of any. It's probably not very popular because managed code is definitely not the best angle to approach this from, but considering this is Windows Phone, it is, as you noted, your only avenue of approach.
Good luck!
I've written a basic SL4 application to capture audio data from the microphone using CaptureSource. The trouble is, it's raw PCM output - which means huge and uncompressed.
Given that I need this application to run purely within a SL4 environment, how can I compress the PCM audio data into something that can be delivered to a remote server more easily?
Essentially I need a solution that I can also deploy/include in a Windows Phone Series 7 application as well as one that will work in the browser environment - so managed code solutions only, I think?
In conversation, people have suggested Speex and WMA for instance, but I haven't found any libraries or examples that work without requiring reference to DLL's that won't work in a SL4 project.
Just a small addition to Jason's post:
There is another port of Speex to .Net and Silverlight 4 called NSpeex.
Please see the WavFileHelper class in Silverlight 4 Rough Notes: Camera and Microphone Support on Mike Taulty's blog (a bit lower than the middle of the page, but the full article is worthwhile) in which he compresses the PCM file to WAV.
Here's another example of when writing to WAV you can change values such as Mono/Stereo, which will directly change the size of the WAV file: Audio recorder Silverlight 4 sample. And one more that gives more details about writing to WAV: Creating Sound using MediaStreamSource in Silverlight 3 Beta
Take a look at this. It looks like he has ported the Speex encoder to C# for the exact problem you are trying to solve. It is available here. Speex is designed for speech and should perform better than wma, mp3, or other audio codecs that are designed to handle music if you are just encoding speech, which I assume since you are grabbing from the mic.
This article http://alvas.net/alvas.audio,articles.aspx#how-to-save-audio-to-mp3-on-silverlight about save audio on client. To send audio data to a server you can use WebClient, for example.
You can do encoding thru the server, by send all stream to WCF service and do your encoding thru Microsoft Expression Encoding SDK API.
Please, see this url that i have asked before:
http://forums.silverlight.net/forums/t/181141.aspx
Regards
I have a stream of u-Law compressed PCM data I am extracting from a Camera, I need to play this out the speakers? Anybody know how? I've tried decoding the u-Law into normal WAV Data and then use SoundPlayer but it never seems to work! Always SoundPlayer only supports PCM Data?
I know the sounds ok, because I have saved it to a file (using a custom createWavHeader method) and iTunes can play it.
Windows comes with an ACM codec to convert u-law to PCM. You can use NAudio and use the WaveFileReader and the WaveFormatConversionStream to get a PCM stream you can play easily.
Similarly, there is a freeware library lizPlay providing PCM playing capabilities. Worth considering as developers of lizPlay pay special attention to ensure there is a version that is free of patent infringement issues.