I've written a basic SL4 application to capture audio data from the microphone using CaptureSource. The trouble is, it's raw PCM output - which means huge and uncompressed.
Given that I need this application to run purely within a SL4 environment, how can I compress the PCM audio data into something that can be delivered to a remote server more easily?
Essentially I need a solution that I can also deploy/include in a Windows Phone Series 7 application as well as one that will work in the browser environment - so managed code solutions only, I think?
In conversation, people have suggested Speex and WMA for instance, but I haven't found any libraries or examples that work without requiring reference to DLL's that won't work in a SL4 project.
Just a small addition to Jason's post:
There is another port of Speex to .Net and Silverlight 4 called NSpeex.
Please see the WavFileHelper class in Silverlight 4 Rough Notes: Camera and Microphone Support on Mike Taulty's blog (a bit lower than the middle of the page, but the full article is worthwhile) in which he compresses the PCM file to WAV.
Here's another example of when writing to WAV you can change values such as Mono/Stereo, which will directly change the size of the WAV file: Audio recorder Silverlight 4 sample. And one more that gives more details about writing to WAV: Creating Sound using MediaStreamSource in Silverlight 3 Beta
Take a look at this. It looks like he has ported the Speex encoder to C# for the exact problem you are trying to solve. It is available here. Speex is designed for speech and should perform better than wma, mp3, or other audio codecs that are designed to handle music if you are just encoding speech, which I assume since you are grabbing from the mic.
This article http://alvas.net/alvas.audio,articles.aspx#how-to-save-audio-to-mp3-on-silverlight about save audio on client. To send audio data to a server you can use WebClient, for example.
You can do encoding thru the server, by send all stream to WCF service and do your encoding thru Microsoft Expression Encoding SDK API.
Please, see this url that i have asked before:
http://forums.silverlight.net/forums/t/181141.aspx
Regards
Related
I want to convert a WAV file into a M4A file.
I could not find any reference or any example of how to achieve that.
Naudio should do the trick i assume, but i havent figured out how to do this yet.
I am writing a WebAPI2 project, and i need to return an m4a audio file to the user upon request. Been using a PushStreamContent in order to provide the user with the file, but using an MP3 as the conversion target, lead to many difficulties with browser support and Seek/Pause/Stop support.
Thanks in advance,
Nokky.
M4A files simply contain AAC encoded audio. Have a look at the Media Foundation Encoder demo in the NAudio WPF demo application to see an example of using MediaFoundationEncoder to create AAC files. It will let you encode as aac if you have an appropriate Media Foundation Codec on your machine (you should have with Win 7 and above, although note that it doesn't do low bitrates).
I am developing application that receives media content(.mp3/.mp4/.avi) in form of bytes.
However, as WPF doesn't support playing media from stream, So I started with WMP.dll.
I am creating class library that receives media in form of bytes and media format and play media accordingly.
Referring Creating the WMP Programmatically, however, I didn't find method to pass bytes stream.
So My question is how do I play media from bytes using wmp.dll?
Edit: I am using WMP.dll using COM located at C:\WINDOWS\system32\wmp.dll
I've been through this process a long time ago ...
Basically I would advise you NOT to go down the WMP route at all. It's heavy, cumbersome and not very nice to work with. I encountered lots of issues along the way basically. Least of all it being feature rich (which it isn't).
The best solution I found and the one I'm still using now is a library called BASS from Un4SeenDevelopments.
This library is tiny < 100k and basically it's awesome. Never had a problem with it and it has it's own .NET wrapper that is a free download from the site.
The support is amazing and the compatibility via a massive selection of plugins and additional libraries is staggering.
Highly recommended for what you want to do.
"BASS is an audio library for use in software on several platforms. Its purpose is to provide developers with powerful and efficient sample, stream (MP3, MP2, MP1, OGG, WAV, AIFF, custom generated, and more via OS codecs and add-ons), MOD music (XM, IT, S3M, MOD, MTM, UMX), MO3 music (MP3/OGG compressed MODs), and recording functions. All in a compact DLL that won't bloat your distribution."
I have a device that provides an H.264 video stream from a URL like:
rtsp://192.168.0.10:554/videoservice
Since this is live video I don't need to be able to control it (pause, rewind, etc), just play. Is this supported by MediaElement or another standard class, do I need something like Smooth Streaming Client SDK or is this a lot more complicated than I thought?
Update:
I downloaded Microsoft's Player Framework but this doesn't play the stream either. I can't find anything in the examples about RTSP.
Update:
I used Wireshark to compare the packets that VLC Media Player (which works) sends with MediaElement and Player Framework and neither of them seemed to use RTSP protocol. Instead they're sending WPAD packets to a different address, even though I've provided the IP address of the source. Why is this necessary? Is there any way of turning this behaviour off?
See the post here. You need to wrap your data in MPEG-4 Part 14 containers and then pass them into MediaElement.SetSource. It appears you can't do this just using the documented API. Here is the important information from the link:
We don't support RTP but rather the MPEG-4 Part 14 container format.
You will need to write our own source to be able to parse the data a
pass it directly to the Microsoft decoder. At this time we do not have
any samples on how to write a custom source and have it loaded from
your Metro style app. Unfortunately it is not possible to simply use
the documentation to figure out how to do this. I have been talking
with Stan and we are trying to figure out how and when we can make
this information available. As soon as this information is available
I will announce it on my blog http://blogs.msdn.com/mediasdkstuff/.
Here is a list of supported video formats.
There is also an example here of how to extend the media class which might have an example of how to do something similar to what you are asking.
There is a similar example here.
We are trying to find out any technologies/libraries available in .NET stack (even wrappers on top of 3rd party dlls) that'll help us to build an app that can
1 - Capture an image from a user's video device
2 - Upload it realtime to a server
3 - Process the video (in the server) - eg: Adding a watermark to the video
4 - Stream it back to the user/other users
Preferably, the time delay/latency between step2 and 4 should be minimal
The first requirement (capturing) seems pretty straight forward. The challenge is identifying a suitable way to do the upload, do the processing, and stream it back. Any valid suggestions or ideas?
Recently came acrsoss FFmpeg library, and it has a C# wrapper. Does FFmpeg can be used to do the processing side?
I would go about it this way:
Use silverlight or flash to capture the video camera input, e.g. as detailed here.
You can send the byte-stream over a socket that your server is listening to.
On the receiving end, just use the socket-accepting program as a router-program with a number of listening workers connected. Between workers and router-program, e.g. AMQP with RabbitMQ. Send asynchronous messages (e.g. with reactive extensions) with e.g. the stream encoding to the rabbit-node, which then can either further all messages to one single computer as a part of a conversation/user-session, or interleave between the available workers. Here's the manual. As the video is encoded, it is streamed asynchronously over the message bus back. According to intel tests the bus itself should work well at high throughputs, but they had to use the interleaved tcp channel mode (they tested on a gigabit lan). Other users here have suggested FFlib. You might also look into having the workers convert into webM, but if FFlib works, that might be a lot easier. Each worker publishes over AMQP the next encoded video piece. A server-running program, e.g. the router program I talked about before, starts sending to the client (see no. 4)
Have a client-program, e.g. silverlight/flash connect (for example over the same socket that you opened for client->server data, or over HTTP), and read the byte-stream with a decoder. Render the output.
VideoLab from Mitov can accomplish all of this and is free for personal use (not so free for commercial use, but pricing is not too heavy).
I have bought and use the Delphi version and know it works extremely well, so I'm pretty sure the .NET version will do what you need.
This kind of task is not trivial (as seen by the lack of responses here), so expect to struggle considerably with DirectX/Microsoft Media Encoder- but with this toolkit and some help from the author, you will eventually succeed.
http://www.mitov.com/html/videolab.html
It seems that Splicer can process static video and convert it - I'm not sure about processing a realtime uploaded video - http://splicer.codeplex.com/
Take a look at Video.Show by Vertigo. It's an open source website for user-generated video content. It uses the Expression Encoder to handle compression/video editing. It's not exactly what you need, but it's a good start!
You could use Silverlight for capture as is mentioned above, and then use Expression Encoder to push it to a stream server or stream from there directly.
It should have everything you need:
Smart encoding/smart recompression for
WMV if the source is also WMV and no
frame operations are performed [4],
cuts editing, serial batch encoding,
Live encoding from webcams and DV
camcorders
Decoding/import format support because
of DirectShow
Smooth streaming (720p+ video using
HTTP) with optimized client
(Silverlight) and server (IIS with
smooth streaming)
WebDAV publishing, publishing plugins
for Silverlight Streaming, Amazon S3
Importing XAML overlays created in
Expression Design and customizing
their timing, animation, opacity,
placement and looping
JavaScript trigger events
Windows Media 11 SDK and VC-1 SDK
integration, native MPEG-2 decoder
Adding captions to videos using SAMI
or W3C Timed Text format
Previewing and comparing encoding
settings in real time
Screen capture
Object model for the encoding engine,
SDK downloadable separately
The question is kind of short on details (is this a web server, what os is the server? etc) but I'll take a stab based on what I think you're trying to do.
One thing you might consider is doing the capture and process at one time. If the user is running your client app, have that do the capture and processing via DirectShow. Then all you need to do is upload the video and you can skip the entire server process. This is assuming that the 'user' is under your control - that this is not some random person out there uploading video, but an employee or someone otherwise trusted.
If this isnt the case, then ffmpeg can certainly be used to watermark video on your server. You dont really need 'wrappers' for it. You can just call it as a command line app from your server application and wait for it to finish.
The process really isnt that complex... its the details that are going to matter (for example - what does 'stream' mean to you? Do you really mean 'stream', or is this via http? Thats a huge topic right there)
I'm looking to develop a Silverlight application which will take a stream of data (not an audio stream as such) from a web server.
The data stream would then be manipulated to give audio of a certain format (G.711 a-Law for example) which would then be converted into PCM so that additional effects can be applied (such as boosting the volume).
I'm OK up to this point. I've got my data, converted the G.711 into PCM but my problem is being able to output this PCM audio to the sound card.
I basing a solution on some C# code intended for a .Net application but in Silverlight there is a problem with trying to take a copy of a delegate (function pointer) which will be the topic of a separate question once I've produced a simple code sample.
So, the question is... How can I output the PCM audio that I have held in a data structure (currently an array) in my Silverlight to the user? (Please don't say write the byte values to a text box)
If it were a MP3 or WMA file I would play it using a MediaElement but I don't want to have to make it into a file as this would put a crimp on applying dynamic effects to the audio.
I've seen a few posts from people saying low level audio support is poor/non-existant in Silverlight so I'm open to any suggestions/ideas people may have.
The simple answer is that there is no support for PCM playback from Silverlight in version 2. So unless you want to write a fully managed PCM to MP3 converter you are stuck. Even then I'm not sure you could get the MediaElement to play from isolated storage.
Is there any chance you could use a web service to perform the conversion?
See also this question:
Where's the sound API in Silverlight? Or, how do I write a music app to run in the browser?
Update: Silverlight 3 supports your custom audio sources. However, it won't let you intercept samples to perform effects on WMA or MP3, presumably for DRM reasons, so you would still potentially need to write your own decoder.
Short answer is use a MediaElement + a MediaStreamSource
Check out these:
http://blogs.msdn.com/gillesk/archive/2009/03/23/playing-back-wave-files-in-silverlight.aspx
http://code.msdn.microsoft.com/wavmss/Release/ProjectReleases.aspx?ReleaseId=2417
Basically, write a decoder in managed code to convert G.711 a-Law to PCM, then do whatever modifications you want to the raw values, then pass those into a MediaStreamSource.
Looks like Silverlight 3 supports direct PCM output now, or will when released. I don't see anything in the docs about the raw AV pipeline yet.
Mark Heath's answer is correct - only certain formats are supported - mp3 and certain flavours of WMA (unfortunately not WMA lossless which would be 'closer' to PCM).
To play PCM data in Silverlight, you could do the following:
* Convert the PCM into mp3 data, and store it in memory.
* Play the mp3 data using the technique presented at ManagedMediaHelpers. The idea here involves a class called Mp3MediaStreamSource (derived from System.Windows.Media.MediaStreamSource) that provides mp3 chunks to a MediaElement to play. The chunks will need to be in a stream, but of course a memory stream will do.
I initially thought you might be able to provide PCM chunks via MediaStreamSource, but this does not work. It's a real shame as it would solve your problem (and the one I was facing - making a Speex audio file player) really easily!