We Keep Coding

IP Camera continuous snapshots vs. video

I'm making a project using c# 2013, windows forms and this project will use an IP camera to display a video for a long time using CGI Commands.
I know from the articles I've read that the return of the streaming video of the IP camera is a continuous multi-part stream. and I found some samples to display the video like this one Writing an IP Camera Viewer in C# 5.0
but I see a lot of code to extract the single part that represents a single image and displays it and so on.
Also I tried to take continuous snap shots from the camera using the following code.
HttpWebRequest req=(HttpWebRequest)WebRequest.Create("http://192.168.1.200/snap1080");
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
Stream strm = res.GetResponseStream();
image.Image = Image.FromStream(strm);
and I repeated this code in a loop that remains for a second and counts the no. of snapshots that were taken in a second and it gives me a number between 88 and 114 snapshots per second
IMHO the first example that displays the video makes a lot of processing to extract the single part of the multi-part response and displays it which may be as slow as the other method of taking a continuous snapshots.
So I ask for other developers' experiences in this issue if they see other difference between the 2 methods of displaying the video. Also I want to know the effect of receiving a continuous multi-part stream on the memory is it safe or will generate an out of memory errors.
Thanks in advance

If you are taking more than 1 jpeg per 1-3 seconds, better capture H264 video stream, it will take less bandwidth and cpu.
Usually mjpeg stream is 10-20 times bigger than the same h264 stream. So 80 snapshots per second is a really big amount.

As long as you dispose of the image and stream correctly, you should not have memory issues. I have done a similar thing in the past with an IP Camera, even converting all the images that I take as a snapshot back into a video using ffmpeg (I think it was).

Capture a frame from a video (mpg, wmv, etc) at a specific time interval

Is there a way I can capture a single frame from a video file (mpg, wmv, flv, etc) at a specific point in the video (e.g. 5 seconds or the 25th frame)? Then save it as an image?
[Edit]
Something link YouTube does. That can't be all done manually? ;)

I'd use DirectShow.NET, because it'll let you do a lot of the work in managed code, which is quite a bit more friendly than doing it in native code.
You'll have to construct a filter graph to render the file you want, and you'll also need a file reader for the format of the file (i.e. if it's an MP4 file, you'll need an MP4 demux), and you'll need a decoder for the format of video (i.e. if it's H264, you'll need an H264 decoder filter). I'd use Windows7 if possible, it has much better media support.
Your graph should look something like:
File Reader -> Video Decoder -> Sample Grabber -> Null Renderer
You'll construct your graph, and then call IMediaSeeking to seek to the approximate time of the sample you want. Then run the graph. The decompressed frames will come in through a Sample Grabber callback interface. You can check the timestamps and get the one that's closest to what you need.
From there, you can use .NET to save it as whatever image format you like (JPEG is probably best).

FFMPEG and .net is your best choice

How to read using C++ (C#) sound stream sent by flash?

I need to read sound stream sent by flash audio in my C++ application (C++ is not a real limitation, it may be C# or any other desktop language).
Now flash app sends audio to another flash app but I need to receive the same audio by desktop application.
So, is there a standard or best way how to do it?
Thank you for your answers.

How is the sound actually sent? Via the network?
Edit: You'd be either capturing the audio from an HTTP stream, or an RTMP stream. Run Wireshark to find out, but I suspect you're doing something slightly shady...

You could try using the sound system from the Gnash project.

So basically you want to connect to RTMP sound stream from flash media server from an arbitrary non-flash application? Have you taken a look at http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol ?

Unfortunately, Adobe IS relatively proprietary (hence the apple-adobe wars happening lately), but for several languages, there are projects to help out with RTMP.
WebOrb is commercial, for .NET, Java, PHP:
http://www.themidnightcoders.com/products.html
FluorineFX is open source for .NET only:
http://www.fluorinefx.com/
I haven't used either myself for RTMP, but I have used FluorineFX to connect to a flash remoting (AMF) gateway. I imagine it may do what you need for receiving the audio stream from a .NET-enabled client.

Getting the frames, frame rate and other attributes of video clip
If you have experience with writing applications in Microsoft DirectShow Editing Services (codename Dexter), this will sound very familiar to you. In the Windows environment, traditionally capturing still frames has been done using C++ and Dexter Type Library to access DirectShow COM objects. To do this in .NET Framework, you can make an Interop assembly of DexterLib which is listed under COM References in VS 2005. However it takes you a good amount of work to figure out how to convert your code from C++ to C# .NET. The problem occurs when you need to pass in a pointer reference as an argument to a native function, CLR does not directly support pointers as the memory position can change after each garbage collection cycle. You can find many articles on how to use DirectShow on the CodeProject or other places and we try to keep it simple. Here our goal is to convert a video file into an array of Bitmaps and I tried to keep this as short as possible, of course you can write your own code to get the Bitmaps out of a live stream and buffer them shortly before you send them.
Basically we have two option for using the DirectShow for converting our video file to frames in .NET:
Edit the Interop assembly and change the type references from pointer to C# .NET types.
Use pointers with unsafe keyword.
We chose the unsafe (read fast) method. It means that we extract our frames outside of .NET managed scope. It is important to mention that managed does not always mean better and unsafe does not really mean unsafe!
MediaDetClass mediaClass = new MediaDetClass();
_AMMediaType mediaType;
... //load the video file
int outputStreams = mediaClass.OutputStreams;
outFrameRate=0.0;
for (int i = 0; i < outputStreams; i++)
{
mediaClass.CurrentStream = i;
try{
//If it can the get the framerate, it's enough,
//we accept the video file otherwise it throws an exception here
outFrameRate = mediaClass.FrameRate;
.......
//get the attributes here
.....
}catch
{ // Not a valid meddia type? go to the next outputstream }
}
// No frame rate?
if (outFrameRate==0.0)
throw new NotSupportedException( " The program is unable" +
" to read the video file.");
// we have a framerate? move on...
...
//Create an array to hold Bitmaps and intilize
//other objects to store information...
unsafe {
...
// create a byte pointer to store the BitmapBits
...
while (currentStreamPos < endPosition)
{
mediaClass.GetBitmapBits(currentStreamPos, ref bufferSize,
ref *ptrRefFramesBuffer,
outClipSize.Width, outClipSize.Height);
...
//add frame Bitmap to the frameArray
...
}
}
...
Transfer extracted data over HTTP
So far we have converted our video to an array of Bitmap frames. The next step is to transfer our frames over HTTP all the way to the client�s browser. It would be nice if we could just send our Bitmap bits down to the client but we cannot. HTTP is designed to transport text characters which mean your browser only reads characters that are defined in the HTML page character set. Anything else out of this encoding cannot be directly displayed.
To accomplish this step, we use Base64 encoding to convert our Bitmap to ASCII characters. Traditionally, Base64 encoding has been used to embed objects in emails. Almost all modern browsers including Gecko browsers, Opera, Safari, and KDE (not IE!) support data: URI scheme standard to display Base64 encoded images. Great! Now, we have our frames ready to be transferred over HTTP.
System.IO.MemoryStream memory = new System.IO.MemoryStream();
while (currentStreamPos < endPosition)
{
...
// Save the Bitmpas somewhere in the (managed) memory
vdeoBitmaps.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
//Convert it to Base64
strFrameArray[frameCount] = System.Convert.ToBase64String(memory.ToArray());
//Get ready for the next one
memory.Seek(0, System.IO.SeekOrigin.Begin);
}
memory.Close();
...
But we cannot just send out the encoded frames as a giant string. We create an XML document that holds our frames and other information about the video and then send it to the client. This way the browser can receive our frames as a DOM XML object and easily navigate through them. Just imagine how easy it is to edit a video that is stored in XML format:
14.9850224700412
{Width=160, Height=120}
6.4731334
/9j/4AAQSkZJRgABAQEAYAB....
....
This format also has its own drawbacks. The videos that are converted to Base64 encoded XML files are somewhere between 10% (mostly AVI files) to 300 % or more (some WMV files) bigger than their binary equivalent.
If you are using an XML file, you even don't need a web server , you can open the HTML from a local directory and it should work! I included an executable in the article's download file that can convert your video file to XML document which later can be shown in the browser. However using big files and high resolution videos is not a good idea!
OK, now we can send out our �Base64 encoded video� XML document as we would do with any other type of XML files. Who says XML files always have to be boring record sets anyway

IStreamBufferMediaSeeking setPosition doesn't seem to render

I have created two DirectShow graphs. One captures from a Hauppauge HD-PVR and stores it in a StreamBufferSink. The second one uses a StreamBufferSource, sends the output to an MPEG-2 Demultiplexer, sending the video to the ArcSoft Video Decoder and on to a Video Mixing Renderer 9 set up in windowless mode.
This all works fine for previewing the data. When I use the IStreamBufferMediaSeeking.SetPositions method (getting the interface from the StreamBufferSource) to change the playback position, if I set it anywhere but at the beginning of the stream, the video freezes and stops updating. Calling GetCurrentPosition on IStreamBufferMediaSeeking shows the position is moving on the stream, but the video just doesn't follow along.
I am using C# and DirectShowLib-2005 for programming things.
Any ideas on what is wrong or how to figure out what is going wrong?

What I have discovered is the StreamBufferSink/StreamBufferSource only understand MPEG-2 or DV video. H.264 is not supported so it doesn't know how to seek within the stream and thus I cannot use this component for what I want to do unless I transcode my stream to MPEG-2 which defeats the purpose for having an H.264 stream in the first place.

Further information: This actually will work under Windows 7 with the updates to the Streaming Buffer Engine. To get rewind, I had to demux the stream and add the MPEG-2 Video Stream Analyzer filter before putting the data into the Stream Buffer Sink.

Creating an MJPEG video stream in c#

I have images being sent to my database from a remote video source at about 5 frames per second as JPEG images. I am trying to figure out how to get those images into a video format so I can stream a live video feed to Silverlight.
It seems to make sense to create a MJPEG stream but I'm having a few problems. Firstly I was trying to stream via an HTTP request so I didn't have a deal with sockets but maybe this is breaking my code.
If I try surf to my stream from QT I get a video error, Media player shows the first frame image and Silverlight crashes :)
Here is the code that streams - since I content type used this way can only be sent once I know that it isn't ideal and might be the root cause. All images are coming in via a LINQ2SQL object.
I did already try simply updating the image source of an image control in Silverlight but the flicker isn't acceptable. If Silverlight doesn't support MJPEG then no point even continuing but it looks like it does. I do have access to the h.264 frames coming in but that seemed more complicated via MP4.
Response.Clear();
Response.ContentType = "multipart/x-mixed-replace; boundary=--myboundary";
ASCIIEncoding ae = new ASCIIEncoding();
HCData data = new HCData();
var videos = (from v in data.Videos
select v).Take(50); // sample the first 50 frames
foreach (Video frame in videos)
{
byte[] boundary = ae.GetBytes("\r\n--myboundary\r\nContent-Type: image/jpeg\r\nContent-Length:" + frame.VideoData.ToArray().Length + "\r\n\r\n");
var mem = new MemoryStream(boundary);
mem.WriteTo(Response.OutputStream);
mem = new MemoryStream(frame.VideoData.ToArray());
mem.WriteTo(Response.OutputStream);
Response.Flush();
Thread.Sleep(200);
}
Thanks!
EDIT: I have the stream working in firefox so if I surf to the page I see video! but nothing else accepts the format. Not IE, SL, Media player - nothing.

I did MJPEG a long time ago (3-4 years ago) and I'm scratching my head trying to remember the details and I simply can't. But, if its possible, I would suggest finding some kind of web site that streams MJPEG content and fire up wireshark/ethereal and see what you get over the wire. My guess is you are missing some required HTTP headers that firefox is little more forgiving about.
If you can't find a sample MJPEG stream over the internet, a lot of web cams have software that give you an MJPEG stream. The app I worked on it with was a console for multiple security cameras, so I know that is a common implementation for cams of all types (if they support a web interface).

I'm far from being an expert in MJPEG streaming, but looking at the source of mjpg-streamer on sourcefourge I think you should send each frame separately, writing the boundary before and after each of them. You should of course not write the content-type in the closing boundary.

First, write your mjpeg frames out to separate files. You should then be able to open these in Phototshop (this will independently verify that you are parsing the stream correctly). If this fails, by bet is that you have HTTP headers embedded in your image data.

Have you looked at various web cam setups that exist on the net? A lot of them do some sort of low res update without flicker. You should be able to reverse engineer these types of sites for additional clues to your problem.
Some sites create a GIF animation, maybe that is an option so that the user can see the past minute or so.

About your edit: MJPEG is supported by Firefox and Safari. However other applications do not, like Explorer or Silverlight depending on what you are doing with it.