I have images being sent to my database from a remote video source at about 5 frames per second as JPEG images. I am trying to figure out how to get those images into a video format so I can stream a live video feed to Silverlight.
It seems to make sense to create a MJPEG stream but I'm having a few problems. Firstly I was trying to stream via an HTTP request so I didn't have a deal with sockets but maybe this is breaking my code.
If I try surf to my stream from QT I get a video error, Media player shows the first frame image and Silverlight crashes :)
Here is the code that streams - since I content type used this way can only be sent once I know that it isn't ideal and might be the root cause. All images are coming in via a LINQ2SQL object.
I did already try simply updating the image source of an image control in Silverlight but the flicker isn't acceptable. If Silverlight doesn't support MJPEG then no point even continuing but it looks like it does. I do have access to the h.264 frames coming in but that seemed more complicated via MP4.
Response.Clear();
Response.ContentType = "multipart/x-mixed-replace; boundary=--myboundary";
ASCIIEncoding ae = new ASCIIEncoding();
HCData data = new HCData();
var videos = (from v in data.Videos
select v).Take(50); // sample the first 50 frames
foreach (Video frame in videos)
{
byte[] boundary = ae.GetBytes("\r\n--myboundary\r\nContent-Type: image/jpeg\r\nContent-Length:" + frame.VideoData.ToArray().Length + "\r\n\r\n");
var mem = new MemoryStream(boundary);
mem.WriteTo(Response.OutputStream);
mem = new MemoryStream(frame.VideoData.ToArray());
mem.WriteTo(Response.OutputStream);
Response.Flush();
Thread.Sleep(200);
}
Thanks!
EDIT: I have the stream working in firefox so if I surf to the page I see video! but nothing else accepts the format. Not IE, SL, Media player - nothing.
I did MJPEG a long time ago (3-4 years ago) and I'm scratching my head trying to remember the details and I simply can't. But, if its possible, I would suggest finding some kind of web site that streams MJPEG content and fire up wireshark/ethereal and see what you get over the wire. My guess is you are missing some required HTTP headers that firefox is little more forgiving about.
If you can't find a sample MJPEG stream over the internet, a lot of web cams have software that give you an MJPEG stream. The app I worked on it with was a console for multiple security cameras, so I know that is a common implementation for cams of all types (if they support a web interface).
I'm far from being an expert in MJPEG streaming, but looking at the source of mjpg-streamer on sourcefourge I think you should send each frame separately, writing the boundary before and after each of them. You should of course not write the content-type in the closing boundary.
First, write your mjpeg frames out to separate files. You should then be able to open these in Phototshop (this will independently verify that you are parsing the stream correctly). If this fails, by bet is that you have HTTP headers embedded in your image data.
Have you looked at various web cam setups that exist on the net? A lot of them do some sort of low res update without flicker. You should be able to reverse engineer these types of sites for additional clues to your problem.
Some sites create a GIF animation, maybe that is an option so that the user can see the past minute or so.
About your edit: MJPEG is supported by Firefox and Safari. However other applications do not, like Explorer or Silverlight depending on what you are doing with it.
Related
i am a .Net developer who has written a CMS-System for the Intranet of a specific company.
Our client has the ability to upload videos and other media there and let his employees and customers view them alongside other information.
We use a Standard Httphandler to fetch the uploaded video from HDD of the server and context.response.TransmitFile() it to the Browser.
So we can use this Handler as target for a html5 src-Attribute.
Now i've gotten a request to sort of "emulate" a videostream. The idea is that the client uplaods the video as a file, sets a specific start date from which the video should be viewable and every Request to the video should then return only the slice from the video from startdate to now.
Sort of pretending this video would be a live stream which goes forward on its own.
I tried adapting the HttpHandler to calculate the number of seconds between the startdate and the current request time, multiply it by the bitrate of the video and then simply cut off x bytes from the Stream (for example using Stream.seek) but the resulting data does not get recognized by Brwosers as a valid video stream. I guess this is because of missing (cut away) header-informations and key-frames etc.
Does anybody know a library who allows me to do this (cutting the video in slices without writing them to harddisk, i dont want to have a videofile laying around for every request thats landed on my httphandler)
The video is in mp4 format and i would liek to avoid the additional overhead of having to transcode it (like VLC requires when you use it for streaming)
Any ideas on this topic, im kinda lost...!?
Thanks in advance.
Chris
Clarification:
I do not know how much to cut off the video beforehand, that depends on the moment the stream is requested.
The Formula is easy: Date of the Request (Datetime.Now) - Configured Start time of the video. This Timespan has to be "skipped" from the start of the video.
Ideally i woudl like some library which allows me to load the file as a fileStream, skip x Seconds and write the remaining bytes/frames to the output of the httphandler. But i have no idea how to do this as VLC and FFMPEG seem so only support slicing by writing files, not giving me the sliced data as a stream...
I've recently learned about data streaming and C#. I've practice building a simple server-client Login program like so:
public static TcpClient SocketCLiente = new TcpClient();
NetworkStream _serverStream = default(NetworkStream);
public static string IPServer="127.0.0.1";
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(this._txtUserName.Text + "%" + this.txtPassword.Text + "$");
_serverStream.Write(outStream, 0, outStream.Length);
This works great with text, but what if i want to stream a video to a client? How can I stream a live video from my webcam to a c# application over Lan?
Like the comment mentioned you should check out How can I stream webcam video with C#?. In general all you stream out are bytes. As long as those bytes are formatted in a way that a player, such as flash, or silverlight, or whatever, can understand then it can play the video back.
Now I don't think that post really covers the internet streaming aspect. It really touches more on capturing and playing back locally. Thats what directshow is good at. It abstracts away all the driver nonsense that you used to have to deal with to capture video and audio frame data.
Streaming to the internet, short of rolling your own streaming server, is usually accomplished by hooking into something like red-5, wowza, adobe FMS, rtsp, silverlight streaming etc.
I would use one of those solutions to hook into wherever you are getting your video data from. The gist is that you capture video/audio packets from directshow and then funnel those packets (formatted however the solution wants) to one of these streaming platforms.
Another option, though it is a little more "hackish", would be to stream your video as motion jpeg. This wouldn't let you send audio though. Motion jpeg basically tells the client to keep a connection open and constantly download a new image and replace the old one. It's like a flipbook for web. The advantage here is you can roll your own motion jpeg code pretty easily and it'd be a fun little project.
How i can get stream from Ip Camera, Its using RTP, stream is MPEG4, i have multicast address and port,and i have ip camera's IP address and Port Number. And I cant reach via http forexample (http://ip/jpeg) And I cant reach stream with VLC Player too. forexample (rtp://ipadressofcam:port) and (rtp://multicastaddress:port)
So What is ffmpeg command of that?
I have windows OS, I only write code with C# right now.
But producer created their own ocx which used for viewing cam,that plugin can work on .net but i dont want to use it becouse it doesnt have much funcionality, I mean you cant get current picture or snapshot of cams, thats why i have to do it myself.
Are you sure the stream is not password-protected?
Try to see why your camera rejects VLC requests. And this is how to do it: Install Wireshark, start it, and put a filter for the camera address, like: ip.addr == camera_ip (xxxx.xxxx.xxxx.xxxx). Then, try to connect with VLC, and look there at the messages exchanged. If you see a DROP, UNAUTH, or something that tells you to use a passwd, introduce it. (VLC will first try to connect using an unauthenticated procedure, and if it fails, will ask for a passwd.)
Wireshark will give you clues if the failure reason is something different.
And keep in mind that if VLC can't access it, the chance to find some other way to do it is almost zero. Unless you're a video guru.
Hope it helps!
You can use VLC for such a thing ,and it's ActiveX control which is available for .NET also ,just need to install VLC Media Player and you can set it's control on VS toolbox
http://forum.videolan.org/viewtopic.php?f=32&t=54969
UPDATE
If you are ready to pay for this stuff you can use http://www.mainconcept.com/products/sdks/video.html this Company product's to advance with Decoding and Encoding ,where you can find a huge Library .
So you want to receive video stream from camera and convert individual frame into JPEG file. This sounds good and actually sounds natural: why not? there is a video feed being sent on network and we don't need much, just to pick individual frame.
The original stream is MPEG-4 (Part 2) and desired target encoding is JPEG. This breaks the task into parts of getting MPEG-4 video feed, decoding it into uncompressed images, and encoding into JPEG. Let us go through these from the last one backwards.
JPEG is a popular encoding and there are various codecs with different interface capable of compressing into JPEG. Options include GDI+ API, IJG JPEG library libjpeg, JPEG libraries and codecs for video with DirectShow and other interfaces.
MPEG-4 decoder is a complicated component, however is luckily well standardized and available in different interfaces and from several sources. In particular MPEG-4 Part 2 Video Decoder is shipped as DMO (and also through ) with Windows 7. Decoder is also available as DirectShow filter in ffdshow, Intel Media SDK.
Back to the first step, you need to reach MPEG-4 stream from network. First of all you indicated that the camera is available on multi-cast UDP address. This makes you open a socket and put into onto multi-cast group in order to start receiving RTP data. The data you would receive is not pure MPEG-4 yet, and is formatted according to RTP Payload Format for MPEG-4 Audio/Visual Streams and as you receive RTP stream of messages you will have to be prepared to receive out of order fragments, packet losses etc. You are supposed to receive what you can receive, check the RTP headers, and do your best in reconstructing MPEG-4 feed.
Eventually you need to cover these steps (not necessarily directly, you would rather use third party libraries including those mentioned in other answers) from your application and build a pipeline which stitches together receiving from network and transcoding.
So until now, I saw VLC cant open it but if we create .sdp file and play it with ffplay only a black screen appears.
vlc -vvv dshow:// --sout-keep --sout-all --sout=#rtp{dst=multicastaddress,port=portNo,sdp=file:///C:/test/my.sdp}
Of course this is not pure solution but there is little bit hope.
But obsolute solution can be like that;
There is no way to use just c# to achive what i intend. There are few c++ library components that i can use, but i can use it along with Managed c++ to write interop services and use those dlls in my c# code.Here is what i need:
1-I need an RTP Library, http://research.edm.uhasselt.be/~jori/page/index.php?n=CS.Jrtplib or live555 but live555 has certain limitations.
2-RTP Library is the transport for the MPEG4 stream i pull from my encoder. But i need ways to control the stream, those are provided producer's WSDLs.
3-After that i need a decoder; Libavcodec, I can use libavcodec to convert an Iframe to a jpeg image in .net.
I need to read sound stream sent by flash audio in my C++ application (C++ is not a real limitation, it may be C# or any other desktop language).
Now flash app sends audio to another flash app but I need to receive the same audio by desktop application.
So, is there a standard or best way how to do it?
Thank you for your answers.
How is the sound actually sent? Via the network?
Edit: You'd be either capturing the audio from an HTTP stream, or an RTMP stream. Run Wireshark to find out, but I suspect you're doing something slightly shady...
You could try using the sound system from the Gnash project.
So basically you want to connect to RTMP sound stream from flash media server from an arbitrary non-flash application? Have you taken a look at http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol ?
Unfortunately, Adobe IS relatively proprietary (hence the apple-adobe wars happening lately), but for several languages, there are projects to help out with RTMP.
WebOrb is commercial, for .NET, Java, PHP:
http://www.themidnightcoders.com/products.html
FluorineFX is open source for .NET only:
http://www.fluorinefx.com/
I haven't used either myself for RTMP, but I have used FluorineFX to connect to a flash remoting (AMF) gateway. I imagine it may do what you need for receiving the audio stream from a .NET-enabled client.
Getting the frames, frame rate and other attributes of video clip
If you have experience with writing applications in Microsoft DirectShow Editing Services (codename Dexter), this will sound very familiar to you. In the Windows environment, traditionally capturing still frames has been done using C++ and Dexter Type Library to access DirectShow COM objects. To do this in .NET Framework, you can make an Interop assembly of DexterLib which is listed under COM References in VS 2005. However it takes you a good amount of work to figure out how to convert your code from C++ to C# .NET. The problem occurs when you need to pass in a pointer reference as an argument to a native function, CLR does not directly support pointers as the memory position can change after each garbage collection cycle. You can find many articles on how to use DirectShow on the CodeProject or other places and we try to keep it simple. Here our goal is to convert a video file into an array of Bitmaps and I tried to keep this as short as possible, of course you can write your own code to get the Bitmaps out of a live stream and buffer them shortly before you send them.
Basically we have two option for using the DirectShow for converting our video file to frames in .NET:
Edit the Interop assembly and change the type references from pointer to C# .NET types.
Use pointers with unsafe keyword.
We chose the unsafe (read fast) method. It means that we extract our frames outside of .NET managed scope. It is important to mention that managed does not always mean better and unsafe does not really mean unsafe!
MediaDetClass mediaClass = new MediaDetClass();
_AMMediaType mediaType;
... //load the video file
int outputStreams = mediaClass.OutputStreams;
outFrameRate=0.0;
for (int i = 0; i < outputStreams; i++)
{
mediaClass.CurrentStream = i;
try{
//If it can the get the framerate, it's enough,
//we accept the video file otherwise it throws an exception here
outFrameRate = mediaClass.FrameRate;
.......
//get the attributes here
.....
}catch
{ // Not a valid meddia type? go to the next outputstream }
}
// No frame rate?
if (outFrameRate==0.0)
throw new NotSupportedException( " The program is unable" +
" to read the video file.");
// we have a framerate? move on...
...
//Create an array to hold Bitmaps and intilize
//other objects to store information...
unsafe {
...
// create a byte pointer to store the BitmapBits
...
while (currentStreamPos < endPosition)
{
mediaClass.GetBitmapBits(currentStreamPos, ref bufferSize,
ref *ptrRefFramesBuffer,
outClipSize.Width, outClipSize.Height);
...
//add frame Bitmap to the frameArray
...
}
}
...
Transfer extracted data over HTTP
So far we have converted our video to an array of Bitmap frames. The next step is to transfer our frames over HTTP all the way to the client�s browser. It would be nice if we could just send our Bitmap bits down to the client but we cannot. HTTP is designed to transport text characters which mean your browser only reads characters that are defined in the HTML page character set. Anything else out of this encoding cannot be directly displayed.
To accomplish this step, we use Base64 encoding to convert our Bitmap to ASCII characters. Traditionally, Base64 encoding has been used to embed objects in emails. Almost all modern browsers including Gecko browsers, Opera, Safari, and KDE (not IE!) support data: URI scheme standard to display Base64 encoded images. Great! Now, we have our frames ready to be transferred over HTTP.
System.IO.MemoryStream memory = new System.IO.MemoryStream();
while (currentStreamPos < endPosition)
{
...
// Save the Bitmpas somewhere in the (managed) memory
vdeoBitmaps.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
//Convert it to Base64
strFrameArray[frameCount] = System.Convert.ToBase64String(memory.ToArray());
//Get ready for the next one
memory.Seek(0, System.IO.SeekOrigin.Begin);
}
memory.Close();
...
But we cannot just send out the encoded frames as a giant string. We create an XML document that holds our frames and other information about the video and then send it to the client. This way the browser can receive our frames as a DOM XML object and easily navigate through them. Just imagine how easy it is to edit a video that is stored in XML format:
14.9850224700412
{Width=160, Height=120}
6.4731334
/9j/4AAQSkZJRgABAQEAYAB....
....
This format also has its own drawbacks. The videos that are converted to Base64 encoded XML files are somewhere between 10% (mostly AVI files) to 300 % or more (some WMV files) bigger than their binary equivalent.
If you are using an XML file, you even don't need a web server , you can open the HTML from a local directory and it should work! I included an executable in the article's download file that can convert your video file to XML document which later can be shown in the browser. However using big files and high resolution videos is not a good idea!
OK, now we can send out our �Base64 encoded video� XML document as we would do with any other type of XML files. Who says XML files always have to be boring record sets anyway
I want to provide a choice between streaming 'channels', if you will, in a web application. These are Windows Media streams of live events that are being broadcast from a Windows Media Services 9 distribution network.
I want to provide a relatively recent thumbnail image of the stream (as a user, you would expect to see this), but although I've seen this done in Flash on CNN and countless other sites, I've never seen this done with Windows Media.
I already have a C# / DirectX library that can extract a thumbnail from a WMV file, but obviously the stream doesn't come from a file if it's a live source.
My assumptions so far are:
Will need to run some kind of service application/daemon that will receive a stream into a Windows Media Player object and somehow take thumbnails if WMPlayer supports it...
-or-
Configure the streams to archive to file, and use the existing class library to take a peek at the last frame available in the archive file being written to get the thumbnail.
I would much rather do #1 because it seems like the clean solution, but don't know if/how WMPlayer supports grabbing a frame.
Are there better ways of doing this?
Not sure if using media player is a good idea in such case. I would look into this:
http://sourceforge.net/projects/windowsmedianet/