I have a .Net application which renders graphics using GDI+ and/or XNA and I want to record the frames directly into a video (preferably an AVI, where I can include a codec like xvid etc). I've tried many of the solutions out there, and run into show stoppers with all of them.
All of the FFMPeg based libs seem to be dedicated to transcoding an existing stream, not so much generating a new one from frames.
There is a .Net lib called Splicer on codeplex, but from what I can tell it is more geared towards building a "slideshow" because it takes each frame and stores it on the HD. The directshow solutions behave the same way.
Then there is the AVIFile wrapper, which is almost exactly what I need. The only problem is that when you start a new encoding it pops up (and sometimes UNDER!?) a dialog box. The dialog isn't a problem for normal use, but I also need this to run as a service, so mandatory UI is obviously a show stopper.
Anyone know of another option that is relatively .Net friendly, or am I just asking too much?
I don't know which AVIFile wrapper you're using, but I believe AviFile is probably calling AVISaveOptions to get an initialized AVICOMPRESSOPTIONS struct. You can just initialize AVICOMPRESSOPTIONS yourself. Most members in AVICOMPRESSOPTIONS are pretty easy. lpParms and cbParms contain a block (cbParms = length of block) of binary codec configuration data. You can get this data calling ICGetState.
It should be a fairly simple change to your native AVIFile and your wrapper should still work.
Have a look at this sample for how to init AVICOMPRESSOPTIONS.
Check out this question on the XNA forums for some insight and considerations if you want to do this in XNA. In particular, this answer by the ZMan:
"There's nothing in XNA to help you
here - you need to be looking at other
windows APIs. You would use XNA to
capture the back buffer and then save
each frame out but there's many things
to be concerned about. Pulling each
frame from the back buffer instantly
creates some latency, compressing the
images (if you choose to compress on
the fly) is CPU heavy, saving to a
file adds latency.
DirectShow is one API you can use to
do the compression - there's many
others. Off/MP4 etc. SOund recording
has DSound and I think DShow can use
DSound to grab the audio output too.
They are fairly specialist APIs so you
might want to seek out other forums."
Actually, Splicer can produce smooth videos from multiple still frames, you just need to ensure that you set the apropriate clip offset and clip end parameters when you call ITrack.AddFrame. In my program I use:
videoTrack.AddImage(buffer, 0, 1f / FramesPerSecond);
where videoTrack comes from:
timeline = new DefaultTimeline(framesPerSecond);
videoGroup = timeline.AddVideoGroup("main", framesPerSecond, 24, width, height);
videoTrack = videoGroup.AddTrack();
in the constructor of my video exporter for a simulation program I'm working on.
If you still want to use ffmpeg, then write the frames to disk and use something like
ffmpeg -f image2 -i foo-%03d.jpeg -r 12 -s WxH foo.avi
The syntax foo-%03d.jpeg specifies to use a decimal number composed of three digits padded with zeroes to express the sequence number. It is the same syntax supported by the C printf function, but only formats accepting a normal integer are suitable.
-r 12 specifies 12 frames per second
-s WxH should specify width & height
This is from the ffmpeg documentation.
Related
I'm developing an app which requires me to record a lot of video data but I don't need to store them all.
What I need to do is to keep just the last X minutes of the recorded stream.
This means that I need a way to remove the oldest sample everytime that I need to store a new one.
So I started working with this article: http://msdn.microsoft.com/library/windowsphone/develop/hh394041%28v=vs.105%29.aspx
The first idea that I had was to just call the StopVideoRecording() and then the StartVideoRecording() using a timer each X minutes.
Now, at first this made sense but it won't work.
The problem is that doing this way will delete the previous data each X minutes.
This means that, if we record 12 minutes and we need to keep the last 5, following this idea we'll delete the first 5 and then the second five, leaving just the last 2 minutes and this is not what I was looking for.
I moved then my attention to the VideoSink class because of the OnSample method.
This seems pretty simple, we intercept every sample and we store it in a fixed size byte array (the size depends on the needed length and the sample's size).
When the buffer is full we just shift everything on the left before adding the new sample.
The problem is that a test video of just 1 minute generated something like 2GB of samples and this makes this way really hard to manage.
I know that those samples are uncompressed, but wouldn't be hard, for a smartphone, to get a sample, compress it, shift a big array, insert the sample and write the array to a file and do it on EVERY sample received?
Yeah, I'm talking about writing the array to a file because we need to persist this video somehow. It may happen that the battery stops working, and having it just in RAM will let us loose everything that we recorded!
The last idea that came to mind was to use a combo of VideoSink and FileSink.
While the FileSink does the compression magic (I even decompiled this class to understand what it does but there's no code inside!), we use the VideoSink's OnSample method to manually remove the unneeded data from the mp4 file used by the FileSink.
This one sounds quite hard because I don't know if I can write to the file with both FileSink and VideoSink without concurrency issues, and I've not found a good c# library to help me working with the mp4 files without having to deal with its structure.
The only library that I found is this one http://basemedia.codeplex.com/ but it totally lacks documentation (each link in the documentation page gives a 404 error).
I'm starting to think that this is something that can't be done, but I'd like to see if there's someone here which can point me to the right direction.
EDIT:
Just to be clear, I used the "recording" word and not the "recorded" one beacuse I'm talking about trimming the video while it's still recording!
This is not about editing it once it has been saved, but something more like removing stuff from the stream while I'm writing it to disk.
I cannot provide a code to you but just an idea. Because you have requirements:
I need to do is to keep just the last X minutes of the recorded stream.
Target platform is windows phone 8
I want to add some modifications to your first idea:
Write each minute of video stream in the separate file
Also you need to leave +1 video file more then a number of minutes. For instance if you need 5 minutes you should always keep 6 files because last file may be not full.
By use DirectShow you will able to join this files into one. Be ready to use C++ (As alternative to this you can use some service or make own back-end for this)
I need to create a sound containing tones of many different frequencies. Is there any way to do this in C#?
The only tone generating methods I've seen so far involve console.beep, which works, but only for pure tones (single frequencies).
The Audiere library makes this extremely easy to do. Here's a nearly complete C# program to generate the DTMF tone for the "1" button:
AudioDevice device = new AudioDevice();
OutputStream tone1a = device.CreateTone(697); // part A of DTMF for "1" button
OutputStream tone1b = device.CreateTone(1209); // part B
tone1a.Volume = 0.25f;
tone1b.Volume = 0.25f;
tone1a.Play();
tone1b.Play();
Thread.Sleep(2000);
// when tone1a stops, you can easily tell that the tone was indeed DTMF
tone1a.Stop();
To use Audiere in C#, the easiest way to get up and running is to use Harald Fielker's C# binding (which he claims works on Mono and VS; I can confirm it works in the both full version of VS2005 and using the separate Express 2008 versions of C# and VC++). You'll need to download the Win32 Audiere DLL, lib, and header (which are all in the same zip) and you'll need to build the C# binding from source using both VC++ and C#.
One of the nice benefits of using Audiere is that the calls are non-blocking. You don't have to wait for tone1a to stop playing before you start tone1b, which is clearly a necessity for playing complex tones. I am not aware of any hard upper limits on how many simultaneous output streams you can use, so it's probably whatever your hardware/OS supports. By the way, Audiere can also play certain audio files (MP3, WAV, AIFF, MOD, S3M, XM, IT by itself; Ogg Vorbis, Flac, Speex with external libraries), not just pure generated tones.
One possible downside is that there is a slightly audible "click" as you start or stop an individual tone; it's not noticeable if you add one tone to an already playing tone. The easiest workaround I've found for that is to slowly ramp the tone's volume up or down when you're turning the tone on or off, respectively. You might have to play around with the ramp speed to get it to sound "just right".
Note that Audiere is LGPL-licensed, and the binding has no license attached to it. You'll have to consult your legal team or try to get a hold of Harald if you want to use his binding in a commercial product; or you could just make your own binding and avoid the hassle.
#Tom: Since there is no specific license attached to Harald's library, I'm not sure what implications would come of hosting it; however, I believe I can at least give you fine detail on exactly how my libaudieresharpglue project is set up.
Using Visual C++ Express 2008, open up bindings/csharp/libaudieresharpglue/vc8.0/libaudieresharpglue.sln. VC++ will automatically convert the solution to a VS9 solution.
In another folder, you should have the Audiere package from Sourceforge. Under your VC++ project properties, go to Configuration Properties > C/C++ > General, and make sure you have path/to/audiere-1.9.4-win32/include in your "Additional Include Directories." Then, in that same window, go to Linker > General and make sure you have /path/to/audiere-1.9.4-win32/lib in your "Additional Library Directories." Then, you should be able to build the project (preferably in Release mode) and this output libaudieresharpglue.dll in your vc8.0/Release folder.
Next, open up Visual C# Express 2008. Open up bindings\csharp\test\vc8.0\AudiereCSharpTest.sln and let it convert the solution. The project should build fine, but then you will get an error when you run it. That's fine; in your csharp/test/vc8.0/bin/Release folder, you need to add both libaudieresharpglue.dll from the VC++ solution and audiere.dll from the package from Sourceforge.
Now, you should be able to build and run AudiereCSharpTest. Note that by default, #define stream_test is not commented out at the top of AudiereTest.cs, and that will reference a file that is not on your hard drive. You can simply comment out that #define and uncomment noise_test or square_test.
That should cover it; if I missed any details, hopefully they are small enough to get by on your own :)
You can always try DirectSound...
I have been looking at NAudio with the view to create a program that emulates feedback whilst playing a backing track. There is a blog post about generating sine waves at specific frequencies, I suspect that this could be adapted to do what you are looking for.
Yes it is possible.
Here is a link to a tutorial on this. but of course this also uses Console.Beep
The MSDN documentation doesn't make it clear if Console.Beep is asynchronous or not. If it is, you can probably fire off as many calls as you need in quick succession and nobody will be the wiser. You'd want to use the version that takes a frequency and a duration, of course.
Essentially, you have to implement your own software synthesizer or find a 3rd party library. See: http://en.wikipedia.org/wiki/Demo_(computer_programming)#Music
I am making a program in C# to connect to a webcam and do some image manipulation with it.
I have a working application that uses win32 api (avicap32.dll) to connect to the webcam and send messages to it that sends it to the clipboard. The problem is that, while accessible from paint, reading it from the program results in null pointers.
This is the code I use to connect the webcam:
mCapHwnd = capCreateCaptureWindowA("WebCap", 0, 0, 0, 320, 240, 1024, 0);
SendMessage(mCapHwnd, WM_CAP_CONNECT, 0, 0);
SendMessage(mCapHwnd, WM_CAP_SET_PREVIEW, 0, 0);
And this is what I use to copy the image to the clipboard:
SendMessage(mCapHwnd, WM_CAP_GET_FRAME, 0, 0);
SendMessage(mCapHwnd, WM_CAP_COPY, 0, 0);
tempObj = Clipboard.GetDataObject();
tempImg = (System.Drawing.Bitmap)tempObj.GetData(System.Windows.Forms.DataFormats.Bitmap);
There's some error checking which I have removed from the code to make it shorter.
I've recently started doing some hobby work in this area.
We settled on using the OpenCV library with the opencvdotnet wrapper. It supports capturing frames from a webcam:
using (var cv = new OpenCVDotNet.CVCapture(0))
{
var image = cv.CreateCompatibleImage();
// ...
cv.Release();
}
And if you're doing image manipulation, OpenCV's image processing algorithms have been wrapped within the OpenCVDotNet.Algs assembly.
If you decide to go this route be sure to install OpenCV version 1.0 (and install it to "c:\program files\opencv" if you are on Vista 64-bit, or "mklink OpenCV 'c:\program files (x86)\OpenCV`" from the correct directory or else opencvdotnet will not install).
There are really two ways to get camera data into your application, DirectShow and WIA. Microsoft recommends that you use WIA, and the interface for WIA is fairly simple to wrap your brain around. I created and published an open source WIA desktop library based on work I did a while ago.
Now the problem with WIA in some cases is that it's too simple. For example, if you want to adjust camera properties (like frame rate, resolution, etc) then WIA falls down. Microsoft deprecated DirectShow, but they really didn't give us any replacement that has all of its capabilities, and I've found that it continues to work fine on all existing platforms (it's very entrenched, so I can't imagine support going away any time soon).
There is a very good DirectShow library over at SourceForge. The only "problem" with it is it's really complex and that stems from the fact that DShow is just so damned complex and confusing in the first place. There are lots of things that the wrapper can do that just aren't easy to work out, but they do provide samples for a lot of common use cases like showing video or capturing a frame. If you want to add overlays, or insert other filters, it can do it, but be forewarned that it's not at all straightforward.
Take a look at this article:
http://www.codeproject.com/KB/miscctrl/webcam_c_sharp.aspx
It is way too simpler than installing and using OpenCVNetWrapper.
Have you tried Clipboard.GetImage()? You could also try the various Clipboard.Contains*() methods to see what format the data is stored in the clipboard.
OpenCV capture, EMGU capture and all other capture libraries I have tried
all have the same problem: You cannot go higher than 640x480 programatically
(without opening the windows video source window).
I suggest using this (which does work):
https://github.com/ofTheo/videoInput
This was also asked in How to get web cam images in C#? and you might find the following useful (Sorry for spamming, but this really helps answering the question):
I've just released the complete sourcecode of my Windows app CamTimer (written in .NET/C#). Download/view the complete code (with working Webcam examples) at https://github.com/johanssonrobotics/CamTimer
Happy coding!
I'm using System.Windows.Media.MediaPlayer to play some sounds, and I would like to load these sounds from a ZIP file. It would be nice to be able to load these files as a stream directly from the zip file instead of having to unzip to a temp directory. However, MediaPlayer.open only accepts an URI.
So, is there a way to create a URI that will reference the contents of a stream? Something like an in-memory localhost? Is there any connector from Stream to URI?
System.Media.SoundPlayer should do what you need (you can skip MediaPlayer and URIs altogether). It has a constructor that takes a stream.
Since you need to support polyphony, one approach is to PInvoke into the waveOutXXXX Windows API in order to play the sounds. Here is a good example of how to do this in C#:
https://www.codeproject.com/KB/audio-video/cswavplay.aspx
This example also has code for reading info like duration, sample rate, bits per sample etc.
If you search, you may find claims that the waveOutXXXX API can only play one sound at a time. This was true in Windows 95/98, but is not true any longer.
By the way, SoundPlayer is probably the most frustrating .NET class. The .Play() method automatically stops playback of any other sound played by your process (it might be any other sound played by any .NET app) before starting, which is why you can't do polyphony with it. It would have taken Microsoft's worst intern less than a minute to add a .DontStopJustPlay() method or a StopFirst bool parameter to the .Play() method. It might have taken him until lunch time to add a Duration property.
Although waveOutXXXX is trickier than you would want from a well-designed modern API (and using it in .NET introduces additional problems), the one unmatched advantage it has is that it's come pre-installed on every single Windows computer since Windows 95, including Windows Mobile devices. Every other option (including MediaPlayer) means somebody will always have to install something.
Is there a portable, not patent-restricted way to play compressed sound files in C# / .Net? I want to play short "jingle" sounds on various events occuring in the program.
System.Media.SoundPlayer can handle only WAV, but those are typically to big to embed in a downloadable apllication. MP3 is protected with patents, so even if there was a fully managed decoder/player it wouldn't be free to redistribute. The best format available would seem to be OGG Vorbis, but I had no luck getting any C# Vorbis libraries to work (I managed to extract a raw PCM with csvorbis but I don't know how to play it afterwards).
I neither want to distribute any binaries with my application nor depend on P/Invoke, as the project should run at least on Windows and Linux. I'm fine with bundling .Net assemblies as long as they are license-compatible with GPL.
[this question is a follow up to a mailing list discussion on mono-dev mailing list a year ago]
I finally revisited this topic, and, using help from BrokenGlass on writing WAVE header, updated csvorbis. I've added an OggDecodeStream that can be passed to System.Media.SoundPlayer to simply play any (compatible) Ogg Vorbis stream. Example usage:
using (var file = new FileStream(oggFilename, FileMode.Open, FileAccess.Read))
{
var player = new SoundPlayer(new OggDecodeStream(file));
player.PlaySync();
}
'Compatible' in this case means 'it worked when I tried it out'. The decoder is fully managed, works fine on Microsoft .Net - at the moment, there seems to be a regression in Mono's SoundPlayer that causes distortion.
Outdated:
System.Diagnostics.Process.Start("fullPath.mp3");
I am surprised but the method Dinah mentioned actually works. However, I was thinking about playing short "jingle" sounds on various events occurring in the program, I don't want to launch user's media player each time I need to do a 'ping!' sound.
As for the code project link - this is unfortunately only a P/Invoke wrapper.
I neither want to distribute any
binaries with my application nor
depend on P/Invoke, as the project
should run at least on Windows and
Linux. I'm fine with bundling .Net
assemblies as long as they are
license-compatible with GPL.
Unfortunatly its going to be impossible to avoid distributing binaries, or avoid P/Invoke. The .net class libraries use P/Invoke underneath anyway, the managed code has to communicate with the unmanage operating system API at some point, in order to do anything.
Converting the OGG file to PCM should be possible in Managed code, but because there is no Native Support for Audio in .net, you really have 3 options:
Call an external program to play the sound (as suggested earlier)
P/Invoke a C module to play the sound
P/Invoke the OS APIs to play the sound.
(4.) If you're only running this code on windows you could probably just use DirectShow.
P/Invoke can be used in a cross platform way
http://www.mono-project.com/Interop_with_Native_Libraries#Library_Names
Once you have your PCM data (using a OGG C Lib or Managed Code, something like this http://www.robburke.net/mle/mp3sharp/ of course there are licencing issues with MP3), you will need a way to play it, unfortunatly .net does not provide any direct assess to your sound card or methods to play streaming audio. You could convert the ogg files to PCM at startup, and then use System.Media.SoundPlayer, to play the wav files generated. The current method Microsoft suggests uses P/Invoke to access Sound playing API in the OS http://msdn.microsoft.com/en-us/library/ms229685.aspx
A cross platform API to play PCM sound is OpenAL and you should be able to play (PCM) sound using the c# bindings for OpenAL at www.taoframework.com, you will unfortunatly need to copy a number of DLL and .so files with your application in order for it to work when distributed, but this is, as i've explained earlier unavoidable.
Calling something which is located in 'System.Diagnostics' to play a sound looks like a pretty bad idea to me. Here is what that function is meant for:
//
// Summary:
// Starts a process resource by specifying the name of a document or application
// file and associates the resource with a new System.Diagnostics.Process component.
//
// Parameters:
// fileName:
// The name of a document or application file to run in the process.
//
// Returns:
// A new System.Diagnostics.Process component that is associated with the process
// resource, or null, if no process resource is started (for example, if an
// existing process is reused).
//
// Exceptions:
// System.ComponentModel.Win32Exception:
// There was an error in opening the associated file.
//
// System.ObjectDisposedException:
// The process object has already been disposed.
//
// System.IO.FileNotFoundException:
// The PATH environment variable has a string containing quotes.
i think you should have a look a fmod, which is the mother of all audio api
please feel free to dream about http://www.fmod.org/index.php/download#FMODExProgrammersAPI
The XNA Audio APIs work well in .net/c# applications, and work beautifully for this application. Event-based triggering, along with concurent playback of multiple sounds. Exactly what you want. Oh, and compression as well.
Well, it depends on a patent-related laws in a given country, but there is no way to write a mp3 decoder without violating patents, as far as i know. I think the best cross-platform, open source solution for your problem is GStreamer. It has c# bindings, which evolve rapidly. Using and building GStreamer on Windows is not an easy task however. Here is a good starting point. Banshee project uses this approach, but it is not really usable on windows yet (however, there are some almost-working nightly builds). FMOD is also a good alternative. Unfortunately, it is not open source and i find that its API is somehow C-styled.
There is a pure C# vorbis decoder available that is open source:
http://anonsvn.mono-project.com/viewvc/trunk/csvorbis/
Not sure if this is still relevant. Simplest solution would be to use NAudio, which is a managed open source audio API written in C#. Another thing to try would be utilizing ffmpeg, and creating a process to ffplay.exe (the right binaries are under shared builds).
There is no way for you to do this without using something else for your play handling.
Using the System.Diagnostic will launch an external software and I doubt you want that, right? You just want X sound file to play in the background when Y happens in your program, right?
Voted up because it looks like an interesting question. :D