Webcam usage in C# - c#

I am making a program in C# to connect to a webcam and do some image manipulation with it.
I have a working application that uses win32 api (avicap32.dll) to connect to the webcam and send messages to it that sends it to the clipboard. The problem is that, while accessible from paint, reading it from the program results in null pointers.
This is the code I use to connect the webcam:
mCapHwnd = capCreateCaptureWindowA("WebCap", 0, 0, 0, 320, 240, 1024, 0);
SendMessage(mCapHwnd, WM_CAP_CONNECT, 0, 0);
SendMessage(mCapHwnd, WM_CAP_SET_PREVIEW, 0, 0);
And this is what I use to copy the image to the clipboard:
SendMessage(mCapHwnd, WM_CAP_GET_FRAME, 0, 0);
SendMessage(mCapHwnd, WM_CAP_COPY, 0, 0);
tempObj = Clipboard.GetDataObject();
tempImg = (System.Drawing.Bitmap)tempObj.GetData(System.Windows.Forms.DataFormats.Bitmap);
There's some error checking which I have removed from the code to make it shorter.

I've recently started doing some hobby work in this area.
We settled on using the OpenCV library with the opencvdotnet wrapper. It supports capturing frames from a webcam:
using (var cv = new OpenCVDotNet.CVCapture(0))
{
var image = cv.CreateCompatibleImage();
// ...
cv.Release();
}
And if you're doing image manipulation, OpenCV's image processing algorithms have been wrapped within the OpenCVDotNet.Algs assembly.
If you decide to go this route be sure to install OpenCV version 1.0 (and install it to "c:\program files\opencv" if you are on Vista 64-bit, or "mklink OpenCV 'c:\program files (x86)\OpenCV`" from the correct directory or else opencvdotnet will not install).

There are really two ways to get camera data into your application, DirectShow and WIA. Microsoft recommends that you use WIA, and the interface for WIA is fairly simple to wrap your brain around. I created and published an open source WIA desktop library based on work I did a while ago.
Now the problem with WIA in some cases is that it's too simple. For example, if you want to adjust camera properties (like frame rate, resolution, etc) then WIA falls down. Microsoft deprecated DirectShow, but they really didn't give us any replacement that has all of its capabilities, and I've found that it continues to work fine on all existing platforms (it's very entrenched, so I can't imagine support going away any time soon).
There is a very good DirectShow library over at SourceForge. The only "problem" with it is it's really complex and that stems from the fact that DShow is just so damned complex and confusing in the first place. There are lots of things that the wrapper can do that just aren't easy to work out, but they do provide samples for a lot of common use cases like showing video or capturing a frame. If you want to add overlays, or insert other filters, it can do it, but be forewarned that it's not at all straightforward.

Take a look at this article:
http://www.codeproject.com/KB/miscctrl/webcam_c_sharp.aspx
It is way too simpler than installing and using OpenCVNetWrapper.

Have you tried Clipboard.GetImage()? You could also try the various Clipboard.Contains*() methods to see what format the data is stored in the clipboard.

OpenCV capture, EMGU capture and all other capture libraries I have tried
all have the same problem: You cannot go higher than 640x480 programatically
(without opening the windows video source window).
I suggest using this (which does work):
https://github.com/ofTheo/videoInput

This was also asked in How to get web cam images in C#? and you might find the following useful (Sorry for spamming, but this really helps answering the question):
I've just released the complete sourcecode of my Windows app CamTimer (written in .NET/C#). Download/view the complete code (with working Webcam examples) at https://github.com/johanssonrobotics/CamTimer
Happy coding!

Related

Porting old Flash games using Windows Forms - How to Embed Flash Player dll?

I developed some somewhat popular Flash games around a decade ago, and recently people have been asking me to port them to Steam. I've been researching how to do this for a few days, but it feels like trying to figure out how to get a GameBoy cartridge working on an iPhone, and googling for answers feels like archaeology since nobody talks about Flash anymore.
Flash can build to an exe projector with the Flash Player bundled, but it stores saved data in fragile cookies, so I need to build a wrapper that can exchange data with it to save data more reliably. I've managed to build something simple in Windows Forms (with which I have only limited experience), but there are two possibilities, each with different issues:
One is that I load in the exe win projector version of the Flash file as a Process:
game = Process.Start ( "MARDEK.exe" );
game.WaitForInputIdle ( );
game.EnableRaisingEvents = true;
game.Exited += ( sender, e ) => { Close ( ); };
This works, and doesn't require the user to have Flash Player installed, but I don't know if it can exchange data both ways with C#.
Another possibility is to embed the swf directly:
swf = new AxShockwaveFlashObjects.AxShockwaveFlash ( );
Controls.Add ( swf );
swf.Size = new Size ( 720, 528 );
swf.Location = new Point ( 10, 10 );
swf.BringToFront ( );
swf.LoadMovie ( 0, "MARDEK.swf" );
I think the swf could then use something called ExternalInterface to communicate back and forth with C#. A huge issue though is that I don't think the Flash Player is being bundled, so the user's expected to have that installed separately. That's not acceptable.
I've found some .dll files for the Flash Player installed on my system, but I don't know if it's possible to bundle them in and have the AxShockwaveFlashObjects use them instead of what the user has installed. That'd probably be the ideal situation.
I know this is an obscure issue - which is why I'm having to ask because Google's telling me so little - but any help would be greatly appreciated!
Yo, I've published both AS2 and AS3 Flash games on Steam. Here's what I can remember right away:
The easy way is to make an AS3 Adobe AIR wrapper that loads a swf game file using SWFLoader (if you want to use AIR's Flashplayer, EBF3 uses this) or HTMLLoader (if you want to use the web Flashplayer, EBF5 uses this) and communicates with it using LocalConnection. This works similarly with both AS2 and AS3 swfs. LocalConnection is pretty reliable unless the user's computer is under a lot of stress - then the connection may be closed - so it may be worth checking/remaking it every once in a while.
I'm not actually sure if you need the web Flashplayer, or if AIR's flashplayer is sufficient. Either way, AIR's Flashplayer removes the LOW and MEDIUM stage quality options, which may be a good reason to use the web one if that's a feature you need.
The latest versions of AIR will not include a web Flashplayer in the package (expecting the user to have it installed). You can either use an older version of AIR (AIR 21 or earlier, requires pirating Flash CC2015) to enable a "captive" Flashplayer, or use this obscure work-around in newer versions:
Install NPAPI flash player from https://get.adobe.com/flashplayer/npapi/
Download the latest AIRSDK from https://adobe.com/devnet/air/air-sdk-download.html then remove/rename these files:
\runtimes\air-captive\win\Adobe AIR\Versions\1.0\Resources\NPSWF32.dll
\runtimes\air-captive\win64\Adobe AIR\Versions\1.0\Resources\NPSWF64.dll
And replace them with:
C:\Windows\SysWOW64\Macromed\Flash\NPSWF32_32_0_0_192.dll (paste it in the above location and rename the file to NPSWF32.dll)
And:
C:\Windows\System32\Macromed\Flash\NPSWF64_32_0_0_192.dll (paste it in the above location and rename the file to NPSWF64.dll)
Basically you're just changing some placeholder files to trick the compiler into including Flashplayer. If you publish your AIR application after changing the files, the user won't need to have Flashplayer installed.
You wouldn't be able to figure this out yourself. I got help from Adobe programmers after my complaints went viral on Twitter. It's poorly documented but once you know what to do, it's not a lot of work.
That's the hard stuff done.
You can use the FRESteamWorks() ANE to easily implement Steam achievements, leaderboards, and other features. ( https://github.com/Ventero/FRESteamWorks )
You'll just need to include that in your wrapper source files. The game swf should tell the wrapper when you want to unlock an achievement etc.
As for save data for games, the lazy option would be to keep using Flash sol files. They seem to be less likely to be deleted if they are made by a desktop app. Otherwise you'd have to send your data though LocalConnection (possibly in several parts, I think the limit is 40kb at a time) to the wrapper, which could then create a text file in whatever format you like. If you make the save files in the installation folder, then Steam cloud can automatically back them up without any extra code needed from you.
Another thing to watch out for: The Steam Overlay will freeze if your swf graphics are not being updated. If you have any moments with no animation, you must hide some animation in there so that the overlay keeps updating correctly.
Get in touch if you want more details!

Audio Framework for Mac and Windows

I realize I asked a question similar to this before, but the planning on what I want to do has come some way, and the parameters have become a bit different.
Basically, I'm looking for the best option for decoding and outputting audio on both Mac and Windows. Ideally, there will be no differences in needed code between the two platforms. I just want to be able to pass it a file path or HTTP URL and have it play the audio with the ability to pause, seek, etc.
It must be able to decode MP3 and AAC out of the box with no dependencies on the OS (like Phonon for Qt which is entirely dependent on the OS). Any other codecs beyond that would be a very nice bonus.
I've looked at things like libavcodec, which supposedly can decode about anything, but haven't been able to figure out how to get it to work. So far it seems that libraries I've seen are also ready for Mac and Linux or Windows and Linux but not Mac and Windows.
It does not need to be open source, but if it is needs to be usable in commercial products. I'm OK with licensing something as long as it's not too expensive and easy to use.
Finally, while C/C++ would be preferred, if there's something that would work with C#/Mono, that's OK too.
Any suggestions on something that would work for this?
I've created a C++ audio library for Mac and Windows named "Crosstalk".
Crosstalk is a C++ audio engine that allows you to create and route audio systems in real-time. The engine takes care of all the audio routing and gives you a simple platform for creating system components (E.g. "Mp3 Decoder" component connected to a "Low-Pass Filter" connected to an "Audio Device" and "File Recorder").
It's very easy to use. Here's an example of how to play an mp3 file (These components are provided with the engine):
XtSystem system;
XtMp3Decoder mp3Decoder;
XtAudioDevice audioDevice;
long md = system.addComponent(&mp3Decoder);
long ad = system.addComponent(&audioDevice);
system.connOutToIn(md,0,ad,0);
system.connOutToIn(md,1,ad,1);
mp3Decoder.loadFile("../05 Tchaikovski-Swan Lake-Scene.mp3");
mp3Decoder.play();
Included with Crosstalk is example Xcode and Visual Studio projects.
You can download Crosstalk and check out the API documentation and licensing details here: http://www.adaptaudio.com/Crosstalk
EDIT (01-12-2012):
Crosstalk has been replaced by an open-source project called "DSPatch". DSPatch is essentially an upgraded version of the routing engine behind Crosstalk that is no longer limited to only audio processing. DSPatch allows you to create and route almost any type of process chain imaginable, and free for personal AND proprietary use :)
decode MP3 and AAC out of the box
I'm not aware of any audio library that does this so easilly. The problem is the license issue regarding MP3 decoding.
I discuss some options on this post, and they are good for Windows/Mac OS X, but I'm not sure if they have C# bindings.
If you are willing to write the bindings yourself, you might be interested at libaudiodecoder:
A C++ cross platform MP3/AAC/WMA/WAV decoder.
It comes with an example that shows how to play a song on Windows/Mac through PortAudio.

Create Video, Frame by frame, from .Net

I have a .Net application which renders graphics using GDI+ and/or XNA and I want to record the frames directly into a video (preferably an AVI, where I can include a codec like xvid etc). I've tried many of the solutions out there, and run into show stoppers with all of them.
All of the FFMPeg based libs seem to be dedicated to transcoding an existing stream, not so much generating a new one from frames.
There is a .Net lib called Splicer on codeplex, but from what I can tell it is more geared towards building a "slideshow" because it takes each frame and stores it on the HD. The directshow solutions behave the same way.
Then there is the AVIFile wrapper, which is almost exactly what I need. The only problem is that when you start a new encoding it pops up (and sometimes UNDER!?) a dialog box. The dialog isn't a problem for normal use, but I also need this to run as a service, so mandatory UI is obviously a show stopper.
Anyone know of another option that is relatively .Net friendly, or am I just asking too much?
I don't know which AVIFile wrapper you're using, but I believe AviFile is probably calling AVISaveOptions to get an initialized AVICOMPRESSOPTIONS struct. You can just initialize AVICOMPRESSOPTIONS yourself. Most members in AVICOMPRESSOPTIONS are pretty easy. lpParms and cbParms contain a block (cbParms = length of block) of binary codec configuration data. You can get this data calling ICGetState.
It should be a fairly simple change to your native AVIFile and your wrapper should still work.
Have a look at this sample for how to init AVICOMPRESSOPTIONS.
Check out this question on the XNA forums for some insight and considerations if you want to do this in XNA. In particular, this answer by the ZMan:
"There's nothing in XNA to help you
here - you need to be looking at other
windows APIs. You would use XNA to
capture the back buffer and then save
each frame out but there's many things
to be concerned about. Pulling each
frame from the back buffer instantly
creates some latency, compressing the
images (if you choose to compress on
the fly) is CPU heavy, saving to a
file adds latency.
DirectShow is one API you can use to
do the compression - there's many
others. Off/MP4 etc. SOund recording
has DSound and I think DShow can use
DSound to grab the audio output too.
They are fairly specialist APIs so you
might want to seek out other forums."
Actually, Splicer can produce smooth videos from multiple still frames, you just need to ensure that you set the apropriate clip offset and clip end parameters when you call ITrack.AddFrame. In my program I use:
videoTrack.AddImage(buffer, 0, 1f / FramesPerSecond);
where videoTrack comes from:
timeline = new DefaultTimeline(framesPerSecond);
videoGroup = timeline.AddVideoGroup("main", framesPerSecond, 24, width, height);
videoTrack = videoGroup.AddTrack();
in the constructor of my video exporter for a simulation program I'm working on.
If you still want to use ffmpeg, then write the frames to disk and use something like
ffmpeg -f image2 -i foo-%03d.jpeg -r 12 -s WxH foo.avi
The syntax foo-%03d.jpeg specifies to use a decimal number composed of three digits padded with zeroes to express the sequence number. It is the same syntax supported by the C printf function, but only formats accepting a normal integer are suitable.
-r 12 specifies 12 frames per second
-s WxH should specify width & height
This is from the ffmpeg documentation.

How can I play compressed sound files in C# in a portable way?

Is there a portable, not patent-restricted way to play compressed sound files in C# / .Net? I want to play short "jingle" sounds on various events occuring in the program.
System.Media.SoundPlayer can handle only WAV, but those are typically to big to embed in a downloadable apllication. MP3 is protected with patents, so even if there was a fully managed decoder/player it wouldn't be free to redistribute. The best format available would seem to be OGG Vorbis, but I had no luck getting any C# Vorbis libraries to work (I managed to extract a raw PCM with csvorbis but I don't know how to play it afterwards).
I neither want to distribute any binaries with my application nor depend on P/Invoke, as the project should run at least on Windows and Linux. I'm fine with bundling .Net assemblies as long as they are license-compatible with GPL.
[this question is a follow up to a mailing list discussion on mono-dev mailing list a year ago]
I finally revisited this topic, and, using help from BrokenGlass on writing WAVE header, updated csvorbis. I've added an OggDecodeStream that can be passed to System.Media.SoundPlayer to simply play any (compatible) Ogg Vorbis stream. Example usage:
using (var file = new FileStream(oggFilename, FileMode.Open, FileAccess.Read))
{
var player = new SoundPlayer(new OggDecodeStream(file));
player.PlaySync();
}
'Compatible' in this case means 'it worked when I tried it out'. The decoder is fully managed, works fine on Microsoft .Net - at the moment, there seems to be a regression in Mono's SoundPlayer that causes distortion.
Outdated:
System.Diagnostics.Process.Start("fullPath.mp3");
I am surprised but the method Dinah mentioned actually works. However, I was thinking about playing short "jingle" sounds on various events occurring in the program, I don't want to launch user's media player each time I need to do a 'ping!' sound.
As for the code project link - this is unfortunately only a P/Invoke wrapper.
I neither want to distribute any
binaries with my application nor
depend on P/Invoke, as the project
should run at least on Windows and
Linux. I'm fine with bundling .Net
assemblies as long as they are
license-compatible with GPL.
Unfortunatly its going to be impossible to avoid distributing binaries, or avoid P/Invoke. The .net class libraries use P/Invoke underneath anyway, the managed code has to communicate with the unmanage operating system API at some point, in order to do anything.
Converting the OGG file to PCM should be possible in Managed code, but because there is no Native Support for Audio in .net, you really have 3 options:
Call an external program to play the sound (as suggested earlier)
P/Invoke a C module to play the sound
P/Invoke the OS APIs to play the sound.
(4.) If you're only running this code on windows you could probably just use DirectShow.
P/Invoke can be used in a cross platform way
http://www.mono-project.com/Interop_with_Native_Libraries#Library_Names
Once you have your PCM data (using a OGG C Lib or Managed Code, something like this http://www.robburke.net/mle/mp3sharp/ of course there are licencing issues with MP3), you will need a way to play it, unfortunatly .net does not provide any direct assess to your sound card or methods to play streaming audio. You could convert the ogg files to PCM at startup, and then use System.Media.SoundPlayer, to play the wav files generated. The current method Microsoft suggests uses P/Invoke to access Sound playing API in the OS http://msdn.microsoft.com/en-us/library/ms229685.aspx
A cross platform API to play PCM sound is OpenAL and you should be able to play (PCM) sound using the c# bindings for OpenAL at www.taoframework.com, you will unfortunatly need to copy a number of DLL and .so files with your application in order for it to work when distributed, but this is, as i've explained earlier unavoidable.
Calling something which is located in 'System.Diagnostics' to play a sound looks like a pretty bad idea to me. Here is what that function is meant for:
//
// Summary:
// Starts a process resource by specifying the name of a document or application
// file and associates the resource with a new System.Diagnostics.Process component.
//
// Parameters:
// fileName:
// The name of a document or application file to run in the process.
//
// Returns:
// A new System.Diagnostics.Process component that is associated with the process
// resource, or null, if no process resource is started (for example, if an
// existing process is reused).
//
// Exceptions:
// System.ComponentModel.Win32Exception:
// There was an error in opening the associated file.
//
// System.ObjectDisposedException:
// The process object has already been disposed.
//
// System.IO.FileNotFoundException:
// The PATH environment variable has a string containing quotes.
i think you should have a look a fmod, which is the mother of all audio api
please feel free to dream about http://www.fmod.org/index.php/download#FMODExProgrammersAPI
The XNA Audio APIs work well in .net/c# applications, and work beautifully for this application. Event-based triggering, along with concurent playback of multiple sounds. Exactly what you want. Oh, and compression as well.
Well, it depends on a patent-related laws in a given country, but there is no way to write a mp3 decoder without violating patents, as far as i know. I think the best cross-platform, open source solution for your problem is GStreamer. It has c# bindings, which evolve rapidly. Using and building GStreamer on Windows is not an easy task however. Here is a good starting point. Banshee project uses this approach, but it is not really usable on windows yet (however, there are some almost-working nightly builds). FMOD is also a good alternative. Unfortunately, it is not open source and i find that its API is somehow C-styled.
There is a pure C# vorbis decoder available that is open source:
http://anonsvn.mono-project.com/viewvc/trunk/csvorbis/
Not sure if this is still relevant. Simplest solution would be to use NAudio, which is a managed open source audio API written in C#. Another thing to try would be utilizing ffmpeg, and creating a process to ffplay.exe (the right binaries are under shared builds).
There is no way for you to do this without using something else for your play handling.
Using the System.Diagnostic will launch an external software and I doubt you want that, right? You just want X sound file to play in the background when Y happens in your program, right?
Voted up because it looks like an interesting question. :D

Using C#/WIA version 2.0 on Vista to Scan

I want to implement a paperless filing system and was looking to use WIA with C# for the image acquisition. There are quite a few sample projects on CodeProject, etc. However, after downloading every one of them that I can find, I have run into a problem.
In each and every one of them, the reference to WIALib is broken. When I go to add "Microsoft Windows Image Acquisition" as a reference, the only version available on my development workstation (also the machine that will run this) is 2.0.
Unfortunately, every one of these sample projects appear to have been coded against 1.x. The reference goes in as "WIA" instead of "WIALib". I took a shot, just changing the namespace import, but clearly the API is drastically different.
Is there any information on either implementing v2.0 or on upgrading one of these existing sample projects out there?
To access WIA, you'll need to add a reference to the COM library, "Microsoft Windows Image Acquisition Library v2.0" (wiaaut.dll).
add a "using WIA;"
const string wiaFormatJPEG = "{B96B3CAE-0728-11D3-9D7B-0000F81EF32E}";
CommonDialogClass wiaDiag = new CommonDialogClass();
WIA.ImageFile wiaImage = null;
wiaImage = wiaDiag.ShowAcquireImage(
WiaDeviceType.UnspecifiedDeviceType,
WiaImageIntent.GrayscaleIntent,
WiaImageBias.MaximizeQuality,
wiaFormatJPEG, true, true, false);
WIA.Vector vector = wiaImage.FileData;
(System.Drawing)
Image i = Image.FromStream(new MemoryStream((byte[])vector.get_BinaryData()));
i.Save(filename)
Thats a basic way, works with my flatbed/doc feeder. If you need more than one document/page at a time though, there is probably a better way to do it (from what I could see, this only handles one image at a time, although I'm not entirely sure). While it is a WIA v1 doc, Scott Hanselman's Coding4Fun article on WIA does contain some more info on how to do it for multiple pages, I think (I'm yet to go further than that myself)
If its for a paperless office system, you might want also check out MODI (Office Document Imaging) to do all the OCR for you.
Heres how to target WIA 1.0 also so you can ship your app to Windows Xp. Something I was desperately looking for!!
How to develop using WIA 1 under Vista?
Update: I'm adding this separately since its a different answer (a year later). I learnt XP has WIA 1.0 and Vista onward has WIA2.0. You can however install WIA 2.0 for Windows XP Sp1+ from here.
I then also made a small library with code I found somewhere on the interweb here, it also has the ability to scan multiple pages:
http://adfwia.codeplex.com/
It doesn't need to be WIA. I was mostly looking at the WIA setup because it offers the same basic interface for different scanners. I've got 3 scanners on this machine and the TWAIN drivers/software for all of them suck (like blocking the screen during scanning).
For document management, I'm really looking for simple 200dpi grayscale scans, so most of the stuff in the TWAIN drivers is overkill.
That said, asking here was part of my last attempt to figure out how to do it in WIA before moving on to TWAIN.
Another note: You have to download the WIA 2.0 dll from Microsoft.com and then browse to the dll and add it to your project.

Categories