I have a camera sending raw frames to my application and I need to generate a h264 stream from such frames and make it playable via browser with low latency. My idea is to use a webRTC stream in order to keep latency at minimum.
Until now my approach has been the following:
Use FFmpeg to generate a h264/RTSP stream by means of the command
ffmpeg -fflags nobuffer -re -i "frames%05d.bmp" -pix_fmt yuv420p -c:v libx264 -crf 23 -f rtsp rtsp://localhost:8554/mystream
Use RTSP simple server to publish the RTSP stream.
Use RTSPtoWeb to generate a webRTC stream playable by browsers.
Please note input frames are 728x544 bitmaps.
Until now I had no luck: the RTSP stream produced at step (2) is playable by means of VLC but is has problems when played by means of webRTC, e.g. continuous freezes. Please note I can reproduce h264/RTSP stream produced by AXIS IP cameras by means of RTSPtoWeb with no problem.
Furthermore I'd need the frames to be passed to FFmpeg by a C# application instead of reading them from disk.
Of course if you know of a way to directly generate a h264/webRTC stream from an image sequence that would be wounderful.
Has anyone ever tried something like this?
I can capture the camera image by ffmpeg and send it to ffserver, but what happens next with the data? Can I collect it on the other site with some other client that uses ffmpeg (e.g. some c# wrapper for it)? If so - how exactly does the data look like and how can I present it to the final user? Can I just display the data on display port? Or is there some other controller for that?
Thanks!
ffserver takes a feed as input (your camera stream) and outputs a stream based on the specs you put in its configuration file for that particular feed. You decide the output format based on your needs.
You can send to any video player that supports the format you chose.
Example configuration that will output a stream compatible with Windows Media Player:
<Feed feed1.ffm>
File /tmp/feed1.ffm
FileMaxSize 200K
ACL allow 127.0.0.1
</Feed>
# ASF compatible
<Stream test.asf>
Feed feed1.ffm
Format asf
VideoFrameRate 15
VideoSize 352x240
VideoBitRate 256
VideoBufferSize 40
VideoGopSize 30
AudioBitRate 64
StartSendOnKey
</Stream>
You would then access you stream using http://<ffserver_ip_address_or_host_name>:<ffserver_port>/test.asf
In C# you could use a MediaElement to play it. Here's the list of supported formats.
Extra reading:
Streaming media with ffserver
Sample ffserver configuration
I have a simple c# application that record & stream audio with naudio & lame. I use System.Net.Socket for the streaming, just wait for connection, read the header and then pass the socket to lame encoder that will send chunks of mp3.
Problem is, FF's audio tag cannot play the thing, it just returns 'All candidate resources failed to load. Media load paused.' And I specifically mention FF's audio tag because I can play it directly if I put the link in url tab. Searching, I found https://bugzilla.mozilla.org/show_bug.cgi?id=502894#c1 that explain why firefox does all sort of random requests to determine file length and such. But I don't want that, is there anyway of making FF leaves my audio alone? (make it behaves as if I put the link in url tab)
How i can get stream from Ip Camera, Its using RTP, stream is MPEG4, i have multicast address and port,and i have ip camera's IP address and Port Number. And I cant reach via http forexample (http://ip/jpeg) And I cant reach stream with VLC Player too. forexample (rtp://ipadressofcam:port) and (rtp://multicastaddress:port)
So What is ffmpeg command of that?
I have windows OS, I only write code with C# right now.
But producer created their own ocx which used for viewing cam,that plugin can work on .net but i dont want to use it becouse it doesnt have much funcionality, I mean you cant get current picture or snapshot of cams, thats why i have to do it myself.
Are you sure the stream is not password-protected?
Try to see why your camera rejects VLC requests. And this is how to do it: Install Wireshark, start it, and put a filter for the camera address, like: ip.addr == camera_ip (xxxx.xxxx.xxxx.xxxx). Then, try to connect with VLC, and look there at the messages exchanged. If you see a DROP, UNAUTH, or something that tells you to use a passwd, introduce it. (VLC will first try to connect using an unauthenticated procedure, and if it fails, will ask for a passwd.)
Wireshark will give you clues if the failure reason is something different.
And keep in mind that if VLC can't access it, the chance to find some other way to do it is almost zero. Unless you're a video guru.
Hope it helps!
You can use VLC for such a thing ,and it's ActiveX control which is available for .NET also ,just need to install VLC Media Player and you can set it's control on VS toolbox
http://forum.videolan.org/viewtopic.php?f=32&t=54969
UPDATE
If you are ready to pay for this stuff you can use http://www.mainconcept.com/products/sdks/video.html this Company product's to advance with Decoding and Encoding ,where you can find a huge Library .
So you want to receive video stream from camera and convert individual frame into JPEG file. This sounds good and actually sounds natural: why not? there is a video feed being sent on network and we don't need much, just to pick individual frame.
The original stream is MPEG-4 (Part 2) and desired target encoding is JPEG. This breaks the task into parts of getting MPEG-4 video feed, decoding it into uncompressed images, and encoding into JPEG. Let us go through these from the last one backwards.
JPEG is a popular encoding and there are various codecs with different interface capable of compressing into JPEG. Options include GDI+ API, IJG JPEG library libjpeg, JPEG libraries and codecs for video with DirectShow and other interfaces.
MPEG-4 decoder is a complicated component, however is luckily well standardized and available in different interfaces and from several sources. In particular MPEG-4 Part 2 Video Decoder is shipped as DMO (and also through ) with Windows 7. Decoder is also available as DirectShow filter in ffdshow, Intel Media SDK.
Back to the first step, you need to reach MPEG-4 stream from network. First of all you indicated that the camera is available on multi-cast UDP address. This makes you open a socket and put into onto multi-cast group in order to start receiving RTP data. The data you would receive is not pure MPEG-4 yet, and is formatted according to RTP Payload Format for MPEG-4 Audio/Visual Streams and as you receive RTP stream of messages you will have to be prepared to receive out of order fragments, packet losses etc. You are supposed to receive what you can receive, check the RTP headers, and do your best in reconstructing MPEG-4 feed.
Eventually you need to cover these steps (not necessarily directly, you would rather use third party libraries including those mentioned in other answers) from your application and build a pipeline which stitches together receiving from network and transcoding.
So until now, I saw VLC cant open it but if we create .sdp file and play it with ffplay only a black screen appears.
vlc -vvv dshow:// --sout-keep --sout-all --sout=#rtp{dst=multicastaddress,port=portNo,sdp=file:///C:/test/my.sdp}
Of course this is not pure solution but there is little bit hope.
But obsolute solution can be like that;
There is no way to use just c# to achive what i intend. There are few c++ library components that i can use, but i can use it along with Managed c++ to write interop services and use those dlls in my c# code.Here is what i need:
1-I need an RTP Library, http://research.edm.uhasselt.be/~jori/page/index.php?n=CS.Jrtplib or live555 but live555 has certain limitations.
2-RTP Library is the transport for the MPEG4 stream i pull from my encoder. But i need ways to control the stream, those are provided producer's WSDLs.
3-After that i need a decoder; Libavcodec, I can use libavcodec to convert an Iframe to a jpeg image in .net.
Currently we're trying to create a project which has access via web cam to capture
the Video and Audio and store it on Cache.
We have been able to successfully capture the Video into the "Isolated Storage" or "Raw" File format. The next step is to create a silverlight friendly format like WMV version 8 or 9 which can be used for Replay.
Asof now we have been able to successfully get a project from Link
to capture the video and encode it into AVI format for replay mode. Unfortunately we've not been able to get a client version code snippet or API that allows encoding into WMV format directly.
Any pointers in the direction would be most appreciated.
TIA
For the moment there is no easy solution for encoding video/audio raw format from SL, and I think SL5 does not have that capabilities as well.
The only solution I found when I had that question too, was from StreamCoders's SilverSuite product, but it was too pricey for my needs. You might have a look there:
http://www.streamcoders.com/products/silversuite.html
Another solution would be to upload the raw data to a server and perform the encoding there, problem is the size of the raw data file are so big that it make it even not suitable.
I tried to zip the raw file, for a 45 second video/audio I had ~ 320 MB file size and ~ 210 after zipping. That was still way too high for my need.
So far I've not see much hope encoding raw file from the client side apart from the StreamCoders products.
Good luck
We have been developing the same application as a Pure Silverlight Browser solution and moved towards an out of browser solution where -
We capture the RAW Isolated Storage Format and convert it into AVI using AVIDLL available from the following urls -
Silverlight 4 More on Capturing Videos from Webcams
Silverlight 4 Yet More on Capturing Videos from Webcams
And later convert the AVI video (RAW->AVI) which is still large into a WMV file using FFMpeg.exe
Statistics on SIZE:
RAW atleast 700MB for a 1 min recording
AVI atleast 600MB after conversion from RAW
WMV atleast 500KB after conversion from AVI
Statistics on TIMELINE:
RAW to AVI conversion takes about 1min for the 700MB conversion
AVI to WMV conversion takes another 1min for the 600MB conversion
Comparing this with the RAW file upload to the server and converting it would be quite a time consuming process considering that the RAW file size is large.
Any optimizations which could help change the time taken to convert or perform a direct conversion to WMV could make the solution better.
Cheers !