I have searched for this issue, for at least one week and I did not find something helpful.
The question is that I want to grab a frame from a video uploaded using C#. I used some APIs but they only grab frame from avi, mpeg and mpg files but when the client uploads an m4v, 3gp, mov, m2v or mp4 file, the respective APIs found on the web do not support retrieving the frames from these types of videos.
Am I missing something or is there an API that can handle retrieving frames from all types of videos in C#.
Thanks in advance
Solutions to this will vary per video encoding type. You will be hard-pressed to find one system to support them all.
You might consider executing a call to ffmpeg to generate the output you desire as it supports a very large variety of formats. See this question for a few .NET-based ffmpeg wrappers that may be helpful.
Related
I get a stream of frames, an initial SPS and PPS h264 data packet and then packets for the I and P frames.
Using c# .NET I want to convert into a series of JPEGs? Has anyone done this?
I have tried AForge.NET FFMpeg wrapper, but can only go from MP4 file to JPEGs?
Also looked at DirectShow.
I can't seem to find an examples that even come close to doing this?
Thanks
In Directshow, it sounds like you need to introduce a SampleGrabber filter into your filter graph. Insert this after the H264 decoder.
This is a pass through filter which can receive a callback containing each video frame. You can then obviously choose what you do with the frame, is save it to disk as a jpeg etc.
Rather than regurgitate MSDN, there is a great page explaining its usage here:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd407288(v=vs.85).aspx
Update taking account of Roman's comments:
SampleGrabber is deprecated as its part of Directshow Editing Services. However, it is quite self contained. If it was removed from a later version of Windows it would be straight forward to replace with an alternative filter. I still use it in one of my consumer applications. Roman is correct though - it has a steep learning curve.
I've built a Windows Phone class to convert some WriteableBitmap into a AVI Full Frame (Uncompressed). Videos are really huge. Is there a simple codec implementation existing, like a codec that just zip images, or something that is making a next/previous XOR then zip it in jpeg?
Windows Phone doesn't allow any unsafe code and most DLL cannot be wrapped into a C# WP library. So that's why I'm coding something from scratch. Note that I'm more efficient at coding from scratch than in studying C++ existing sources (I'm not a C++ coder), so what I'm searching is infos about a compressed AVI format that can be achieved without writing 100000 lines. I've used AVI because the specs are simple.
[EDIT]
I've found something very interesting here on codeproject, from a 2004 article. It's a 100% C# source to convert frames to mpeg-1. Sadly that's i frame, and not p frame, so files are 3 times larger than an expected mpeg-1 average file size.
[EDIT]
To describe more my project, what I'll do is to apply some effects on a captured movie. This movie will then be uploaded on Youtube or some other websites. Thus, the user expect the exact resolution used on the phone, at least 25 frames/s, a decent quality, and a short upload time. So I can't stop with a Mpeg-1 I-Frames. I'll need to study about prediction in mpeg-1.
I take this to be a continuation/re-post of your previous question (but with a few more details). As I mentioned in the comments of that post, there is a whole universe of video codecs out there. One reason for the proliferation is that a lot of people like to re-invent wheels. However, a more salient reason is that there are a lot of different use cases for video.
You seem to be asking for a lot, yet there are a lot of variables you have not presented:
You express a need for a video encoder that will run purely in software on a Windows Phone device, which is necessarily a fairly low-powered machine; do you need it to run in real-time? I.e., do you expect a frame of video to be compressed almost immediately after to send in the uncompressed frame (within a few milliseconds)? Or can you let the device think about the compression for awhile?
How large are the video frames? Are you doing screen capture on a WP device, i.e., computer-generated data? Or are you reading raw frames from the camera and hoping to compress those?
Following from the previous point, what type of video data? Computer-generated data will look better with a certain class of codecs. Photo-quality images (from camera) implies a different family of codecs.
What bitrate are you aiming for? If you have 1 second of video, what's the max amount of bytes it should occupy (or so you hope)?
Who is the eventual consumer of the video? In the last post, you indicated you wanted to upload to YouTube. If that's the case, you're in luck, since YouTube -- backed by FFmpeg -- handles nearly every codec in the universe, so you would have a lot of options.
I don't know much about Windows Phone programming. However, any WP device is going to technically have hardware video encoding capabilities. I've done some cursory Googling to determine if you get any access to that at the application programming level but I can't find any evidence that it's possible (and this SO answer states that the functionality is not there).
I hope to impress upon you that writing a video encoder is a LOT of work (look at my username; I know from whence I speak). Generally, they require quite a lot of CPU horsepower (and, consequently, battery power, especially when implemented in pure software). However, you have already made some guesses about a codec that uses standard zlib. In fact, there are a few video codecs based on straight zlib, namely MSZH and ZLIB, collectively the Lossless Codec Libraries. That wiki page has a basic bitstream description (disclosure: I operate that wiki site). I'm confident the WP libraries include access to zlib encoding, so this might be a starting point, and YouTube should be able to digest the resulting files.
There is also a video codec that combines XOR and zlib as you guessed (Dosbox Capture Codec), but it's probably not appropriate for your application.
Do the libraries provide access to standard JPEG (i.e., can it encode JPEG files)? Another option (depending on the video type) would be successive frames of still JPEG images stuffed in the AVI file. This is known as Motion JPEG or MJPEG. However, it's roughly equivalent in bitrate to intra-only MPEG-1, which you expressed as being inadequate.
I hope I have given you some ideas and useful avenues to pursue on your path to a solution.
I am developing a windows phone 7 application and it does video recording. I would like to get the sound portion of the video file (MP4) and do some enhancements on the sound. I believe sound is saved as AAC frames in MP4. (Right?) How can I extract sound of a videa MP4 file?
Since this is a video file, it can be huge file. So uploading to cloud and processing there is not a good option. Since this WP7 application I cannot use unmaged dlls :( Is there a way to do in pure C#? Any open source tools/samples?
Thanks!
MP4 is a container format and realistically the sound portion isn't always AAC. It could be MP3 or any other number of different audio formats. You may be thinking of M4A, which I believe requires either AAC or ALAC.
On the subject of audio extraction, it should be possible to extract the audio from an MP4 using just managed code. You'll have to read up on the MP4 format (here, for example - this question is also worth reading) and then search through the file for the location of the audio and then either copy it to its own buffer or do your manipulations in chunks. Even then, you'll have to be able to recognize when it isn't an audio format that your app won't support.
It's possible that there already exists a .net library that can do all of this but I don't know of any. It's probably not very popular because managed code is definitely not the best angle to approach this from, but considering this is Windows Phone, it is, as you noted, your only avenue of approach.
Good luck!
I am currently developing a video conference web based application. My project is based upon the following example : http://www.codeproject.com/Articles/65190/Your-First-Step-to-the-Silverlight-Voice-Video-Cha ..I am currently experiencing a 'lag' in the video conference per se..
First of all i, what could be the cause of such lag? Could it be due to codec reasons? if not what could be the other possibilities?
As regards to the codec, the current project, takes a frame (i.e. JPG) encodes it and sends it.. What other codec are recommended for encoding a JPG frame and decoding it on the receiving end?
Thanks, Any kind of help is highly appreciated..
I have no experience with the work referenced on CodeProject. But, with that said, if the approach is indeed using frame grabs and sending jpg files, then it will be extremely inefficient, resulting in very low frame rates as you describe.
An application like this should rely on a highly efficient encoding mechanism, based on moving video, and not still images. This will create economies based on deltas in the source image, rather than digitizing a new frame from scratch every time.
Additionally. H.264 is not intended as a real-time transport, whereas H.263 is intended for precisely this type of application.
A simple Google search will reveal a wealth of references, including the topic here on stackoverflow - See this
Good luck.
hi
I am developing a video capture application using C#.net. i captured
video through webcam and saved it as a JPEG images then i want to make a
wmv file with those images. how can i do that what are the basic steps needed for that can any body help
I am working on this myself. I have found two ways that may be possible - both require the purchase of an outside library.
The first one looks to be the easiest but costs the most, although it will allow you to use it for free you will just have to deal with a pop up telling you to purchase the library: http://bytescout.com/products/developer/imagetovideosdk/imagetovideosdk_convert_jpg_to_video.html
The other involves using Microsoft Encoder 4. I am still working on this one. You can get the free version here: http://www.microsoft.com/download/en/details.aspx?id=18974
C# doesn't natively support much in the way of sound or video so outside reference assemblies seem to be a necessity.
Right now that is the best help I can offer until I figure it out.