I get a stream of frames, an initial SPS and PPS h264 data packet and then packets for the I and P frames.
Using c# .NET I want to convert into a series of JPEGs? Has anyone done this?
I have tried AForge.NET FFMpeg wrapper, but can only go from MP4 file to JPEGs?
Also looked at DirectShow.
I can't seem to find an examples that even come close to doing this?
Thanks
In Directshow, it sounds like you need to introduce a SampleGrabber filter into your filter graph. Insert this after the H264 decoder.
This is a pass through filter which can receive a callback containing each video frame. You can then obviously choose what you do with the frame, is save it to disk as a jpeg etc.
Rather than regurgitate MSDN, there is a great page explaining its usage here:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd407288(v=vs.85).aspx
Update taking account of Roman's comments:
SampleGrabber is deprecated as its part of Directshow Editing Services. However, it is quite self contained. If it was removed from a later version of Windows it would be straight forward to replace with an alternative filter. I still use it in one of my consumer applications. Roman is correct though - it has a steep learning curve.
Related
I've built a Windows Phone class to convert some WriteableBitmap into a AVI Full Frame (Uncompressed). Videos are really huge. Is there a simple codec implementation existing, like a codec that just zip images, or something that is making a next/previous XOR then zip it in jpeg?
Windows Phone doesn't allow any unsafe code and most DLL cannot be wrapped into a C# WP library. So that's why I'm coding something from scratch. Note that I'm more efficient at coding from scratch than in studying C++ existing sources (I'm not a C++ coder), so what I'm searching is infos about a compressed AVI format that can be achieved without writing 100000 lines. I've used AVI because the specs are simple.
[EDIT]
I've found something very interesting here on codeproject, from a 2004 article. It's a 100% C# source to convert frames to mpeg-1. Sadly that's i frame, and not p frame, so files are 3 times larger than an expected mpeg-1 average file size.
[EDIT]
To describe more my project, what I'll do is to apply some effects on a captured movie. This movie will then be uploaded on Youtube or some other websites. Thus, the user expect the exact resolution used on the phone, at least 25 frames/s, a decent quality, and a short upload time. So I can't stop with a Mpeg-1 I-Frames. I'll need to study about prediction in mpeg-1.
I take this to be a continuation/re-post of your previous question (but with a few more details). As I mentioned in the comments of that post, there is a whole universe of video codecs out there. One reason for the proliferation is that a lot of people like to re-invent wheels. However, a more salient reason is that there are a lot of different use cases for video.
You seem to be asking for a lot, yet there are a lot of variables you have not presented:
You express a need for a video encoder that will run purely in software on a Windows Phone device, which is necessarily a fairly low-powered machine; do you need it to run in real-time? I.e., do you expect a frame of video to be compressed almost immediately after to send in the uncompressed frame (within a few milliseconds)? Or can you let the device think about the compression for awhile?
How large are the video frames? Are you doing screen capture on a WP device, i.e., computer-generated data? Or are you reading raw frames from the camera and hoping to compress those?
Following from the previous point, what type of video data? Computer-generated data will look better with a certain class of codecs. Photo-quality images (from camera) implies a different family of codecs.
What bitrate are you aiming for? If you have 1 second of video, what's the max amount of bytes it should occupy (or so you hope)?
Who is the eventual consumer of the video? In the last post, you indicated you wanted to upload to YouTube. If that's the case, you're in luck, since YouTube -- backed by FFmpeg -- handles nearly every codec in the universe, so you would have a lot of options.
I don't know much about Windows Phone programming. However, any WP device is going to technically have hardware video encoding capabilities. I've done some cursory Googling to determine if you get any access to that at the application programming level but I can't find any evidence that it's possible (and this SO answer states that the functionality is not there).
I hope to impress upon you that writing a video encoder is a LOT of work (look at my username; I know from whence I speak). Generally, they require quite a lot of CPU horsepower (and, consequently, battery power, especially when implemented in pure software). However, you have already made some guesses about a codec that uses standard zlib. In fact, there are a few video codecs based on straight zlib, namely MSZH and ZLIB, collectively the Lossless Codec Libraries. That wiki page has a basic bitstream description (disclosure: I operate that wiki site). I'm confident the WP libraries include access to zlib encoding, so this might be a starting point, and YouTube should be able to digest the resulting files.
There is also a video codec that combines XOR and zlib as you guessed (Dosbox Capture Codec), but it's probably not appropriate for your application.
Do the libraries provide access to standard JPEG (i.e., can it encode JPEG files)? Another option (depending on the video type) would be successive frames of still JPEG images stuffed in the AVI file. This is known as Motion JPEG or MJPEG. However, it's roughly equivalent in bitrate to intra-only MPEG-1, which you expressed as being inadequate.
I hope I have given you some ideas and useful avenues to pursue on your path to a solution.
I’m making an audio synthesizer and I’m having issues figuring out what to use for audio playback. I’m using physics and math to calculate the source waveforms and then need to feed that waveform to something which can play it as sound. I need something that can 1) play the waveforms I calculate and 2) play multiple sounds simultaneously (like holding one key down on a piano while pressing other keys). I’ve done a fair bit of research into this and I can’t find something that does both of those things. As far as I know, I have 5 potential options:
DirectSound. It can take a waveform (a short[]) as a parameter and play it as sound, and can play multiple sounds simultaneously. But it won’t work with .NET 4.5.
System.Media.SoundPlayer. It works with .NET 4.5 and has better quality audio than Direct Sound, but it has to play sound from a .wav file and cannot play multiple sounds at once (nor can multiple instances of SoundPlayer). I ‘trick’ SoundPlayer into working by translating my waveform into .wav format in memory and then send SoundPlayer a MemoryStream of the in-memory .wav file. Could I potentially achieve control over the playback by altering the stream? I cannot append bytes to the stream (I tried) but I could potentially make the stream an arbitrary size and just re-write all the bytes in the stream with the next segment of audio data every time the end of the stream is reached.
System.Windows.Controls.MediaElement. I have not experimented with this yet, but from MSDNs documentation I don’t see a way to send it a waveform in memory without saving it to disk first and then reading it; I don’t think I can send it a stream.
System.Windows.Controls.MediaPlayer. I have not experimented with this either, but the documentation says it’s meant to be used as a companion to some kind of animation. I could potentially use this without doing any real (user-perceivable) animation to achieve my desired effect.
An open source solution. I’m hesitant to use an open source solution as I find they are typically poorly documented and not very maintainable, but I am open to ideas if there is one out there that is well documented and can do what I need.
Can anyone offer me any guidance on this or how to create flexible audio playback?
http://naudio.codeplex.com , without a doubt. Mark is a regular here on SO, the product is well alive, there are good code examples.
It works. We built some great stuff with it.
I have searched for this issue, for at least one week and I did not find something helpful.
The question is that I want to grab a frame from a video uploaded using C#. I used some APIs but they only grab frame from avi, mpeg and mpg files but when the client uploads an m4v, 3gp, mov, m2v or mp4 file, the respective APIs found on the web do not support retrieving the frames from these types of videos.
Am I missing something or is there an API that can handle retrieving frames from all types of videos in C#.
Thanks in advance
Solutions to this will vary per video encoding type. You will be hard-pressed to find one system to support them all.
You might consider executing a call to ffmpeg to generate the output you desire as it supports a very large variety of formats. See this question for a few .NET-based ffmpeg wrappers that may be helpful.
I have a form with a liveview of the tv-signal (from dvb-t stick). I've the sampleproject "DTViewer" from http://directshownet.sourceforge.net/about.html.
Now I try to capture the stream to a movie-file by clicking a button, but how?
I use C# and DirectShow.NET.
I tried to search in many sampleprojcets but these are made for videoinputs not a dvb-t stick with a BDA (Broadcast Driver Architecture) interface.
Help!
Don’t really know what exactly do you mean by a “movie-file”, but I can tell you how to capture the entire MUX (transport stream). Create a graph with a Microsoft DVBT Network Provider, You_Name_It BDA DVBT Tuner, You_Name_It BDA Digital Capture and MPEG-2 Demultiplexer filters. Once you connect them, enumerate all output pins on the MPEG-2 Demultiplexer and render them. Tune the frequency of your choice (put_TuneRequest). At this point everything is ready to run the graph, but don’t run it! Enumerate all filters in the graph. Disconnect all filters except Microsoft DVBT Network Provider, You_Name_It BDA DVBT Tuner and You_Name_It BDA Digital Capture. Remove all these disconnected filters from the graph except the MPEG-2 Demultiplexer (it has to be in the graph although it is not connected). Add Sample Grabber filter and NULL Renderer filter. Connect Digital Capture filter to Sample Grabber and Sample Grabber to NULL Renderer. You can run the graph now. Through the callback in Sample Grabber filter you will receive the entire MUX. Of course, there is still some work to demux the data, but once you do that, you can capture all TV programs in one MUX at once. The easiest way is to capture it in a TS format because the TS is being broadcasted (188 bytes long packets).
It seems to me VLC has BDA support (BDA.c file reference), maybe you can snoop up something from their code?
There is no simple answer to your question. I have started one such project and have found out that there is very little I know about it, so here is little something from my research.
First, you'll have to understand that dvb-t tuner card or stick doesn't give video frames in the classical sense, but the decoding is done in the pc, on the cpu. External card will provide you with compressed data only, as it fetches it from the air.
Next - data that is delivered to you will be in MPEG2 or MPEG4 Transport Stream format. Which is suitable for streaming or broadcasting, not for saving to file. VLC is able to play TS written to the file, but to record a proper video file, you'll have to either transcode the file or repack it to Program Stream. Google it a little, you'll find the differences.
More - one frequency on the air consists of many channels, and that channel packing is called 'mux'. So - from the BDA tuner/capturer you'll get ALL data, and you'll have to demux it manually or let BDA demuxer do it for you.
Hope that's enough info to get you started, I can post you some interesting links when I get to the real keyboard.
Is there a way I can capture a single frame from a video file (mpg, wmv, flv, etc) at a specific point in the video (e.g. 5 seconds or the 25th frame)? Then save it as an image?
[Edit]
Something link YouTube does. That can't be all done manually? ;)
I'd use DirectShow.NET, because it'll let you do a lot of the work in managed code, which is quite a bit more friendly than doing it in native code.
You'll have to construct a filter graph to render the file you want, and you'll also need a file reader for the format of the file (i.e. if it's an MP4 file, you'll need an MP4 demux), and you'll need a decoder for the format of video (i.e. if it's H264, you'll need an H264 decoder filter). I'd use Windows7 if possible, it has much better media support.
Your graph should look something like:
File Reader -> Video Decoder -> Sample Grabber -> Null Renderer
You'll construct your graph, and then call IMediaSeeking to seek to the approximate time of the sample you want. Then run the graph. The decompressed frames will come in through a Sample Grabber callback interface. You can check the timestamps and get the one that's closest to what you need.
From there, you can use .NET to save it as whatever image format you like (JPEG is probably best).
FFMPEG and .net is your best choice