We Keep Coding

UWP C# How to use ChromaKeyEffect on a video stream from MediaCapture and overlay the result on another MediaCapture webcam video stream?

I'm trying to get one video stream (like webcam) with MediaCapture and apply a Chromakey effect on it and overlay the result on another video stream (like another webcam) with MediaCapture too and record the result in a mp4 file during "live".
I've tried with custom video effect. The ChromaKeyEffect works but I've failed having the secondary input stream for put is as background in my effect class.
I've also try with MediaComposition but it doesn't work with MediaCapture. It use only MediaClips witch are not "live".
I don't know how should I do this.
Edit:
In my page
VideoEffectDefinition definition = new VideoEffectDefinition(typeof(AlphaOnMediaCaptureEffect).FullName);
await mediaCapture.AddVideoEffectAsync(definition, MediaStreamType.VideoRecord);
In a AlphaOnMediaCaptureEffect: IBasicVideoEffect
public async void ProcessFrame(ProcessVideoFrameContext context){
using (CanvasBitmap inputBitmap = CanvasBitmap.CreateFromDirect3D11Surface(canvasDevice, context.InputFrame.Direct3DSurface))
using (CanvasRenderTarget renderTarget = CanvasRenderTarget.CreateFromDirect3D11Surface(canvasDevice, context.OutputFrame.Direct3DSurface, 96))
using (CanvasDrawingSession ds = renderTarget.CreateDrawingSession())
{
ChromaKeyEffect chromaKeyEffect = new ChromaKeyEffect
{
Source = inputBitmap,
Color = Color.FromArgb(255, 0, 255, 0),
Feather = true
};
chromaKeyEffect.Tolerance = 0.3f;
CompositeEffect compositeEffect = new CompositeEffect
{
Sources = { otherInputBitmap, chromaKeyEffect }
};
compositeEffect.Mode = CanvasComposite.SourceIn;
ds.DrawImage(compositeEffect);
}
I need another CanvasBitmap like inputBitmap (otherInputBitmap) which contains frames from a second mediacapture. I said "I've failed" because I didn't find a way to code this. I think I wrong in the "process" to obtain this live video overlay on a live video with an ChromaKeyEffect. I'm not sure that IBasicVideoEffect is the best way to do this.

How do I take a picture/photo using a camera on windows phone/desktop

I would like to make an app that takes pictures that I can save to file or display to screen but am having trouble finding where to start.

I would recommend creating a c# Universal Windows App if you are targeting phone or desktop.
To take a picture you can use LowLagPhotoCapture to get a SoftwareBitmap
public async Task<SoftwareBitmap> takePhotoToSoftwareBitmap()
{
//initialize mediacapture with default camera
var mc = new MediaCapture();
await mc.InitializeAsync();
//create low lag capture and take photo
var lowLagCapture = await mc.PrepareLowLagPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Bgra8));
var photo = await lowLagCapture.CaptureAsync();
//convert to displayable format
SoftwareBitmap displayableImage;
using (var frame = photo.Frame)
{
displayableImage = SoftwareBitmap.Convert(photo.Frame.SoftwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
return displayableImage;
}
To see how to use (Save/Display/Edit) a SoftwareBitmap see the Imaging How-To
For more Windows camera documentation see GitHub Samples or MediaCapture How-To

How to access recorded video in WP 8.1 app

I used this instructions to capture a video
http://msdn.microsoft.com/en-us/library/windows/apps/hh452791.aspx
here is the part were the file is getting written to the app storage
var storageFile = await Windows.Storage.KnownFolders.VideosLibrary.CreateFileAsync("cameraCapture.mp4", Windows.Storage.CreationCollisionOption.GenerateUniqueName);
await _mediaCapture.StartRecordToStorageFileAsync(_profile, storageFile);
How I can access the file on an other page of the app.
I want to play this video within a MediaElement (VideoLeinwand)
var stream = ???
VideoLeinwand.AutoPlay = true;
VideoLeinwand.SetSource(stream, ???);
VideoLeinwand.Play();
Thank you very much!

If I understood correctly and you have succeded in recording a file cameraCapture.mp4, then you should be able to play it for example like this:
StorageFile mediafile = await Windows.Storage.KnownFolders.VideosLibrary.GetFileAsync("cameraCapture.mp4");
var stream = await mediafile.OpenAsync(Windows.Storage.FileAccessMode.Read);
VideoLeinwand.SetSource(stream, mediafile.ContentType);
VideoLeinwand.Play();

C# Windows Universal App MediaElement playing AAC-LC/m4a audio data

I am using MediaCapture class to record voice in aac-lc/m4a format. I am trying to play same audio data on MediaElement. It is not playing the m4a audio. However if i record the voice in 'WAV' format, MediaElement is able to play the audio without any issues. I tried with all possible MIME Types for aac-lc/m4a audio.
Here is the player code:
var inMemoryRas = new InMemoryRandomAccessStream();
var writeStream = inMemoryRas.AsStreamForWrite();
await writeStream.WriteAsync(audioData, 0, audioData.Length);
await writeStream.FlushAsync();
inMemoryRas.Seek(0);
mMediaPlayer_.AudioCategory = AudioCategory.ForegroundOnlyMedia;
String mimeType_ = "";
mMediaPlayer_.SetSource(inMemoryRas, mimeType_);
mMediaPlayer_.AutoPlay = true;
mMediaPlayer_.Play();
I tried with following set of mime types, still no help.
audio/mpeg, audio/mp4, audio/aac, video/mp4, audio/m4a
Again if i record the audio in wav format it plays without any issues.
Here is the recorder code:
MediaEncodingProfile recordProfile = null;
recordProfile = MediaEncodingProfile.CreateM4a(Windows.Media.MediaProperties.AudioEncodingQuality.Low);
mRecordingStream_ = new InMemoryRandomAccessStream();
await m_mediaCaptureMgr.StartRecordToStreamAsync(recordProfile, mRecordingStream_);
I appreciate any help on this, Thanks in advance.

WP8/VS2013 MediaLibrary Songs collection is empty

I'm having a play with writing a mp3 player app for WP8, using MediaLibrary to handle the phone's own mp3 collection.
I want to test the result in the phone emulator on VS2013, but when I use the following code:
using (MediaLibrary library = new MediaLibrary())
{
SongCollection songs = library.Songs;
Song song = songs[0];
MediaPlayer.Play(song);
}
The song collection is empty, presumably because VS doesn't have any knowledge of a media library with songs in.
Is there any way to test this in the emulator using a fake medialibrary or for VS to use windows' media library? I just want to see (or hear) the code working before I proceed :)

I have managed to find a workaround!
If you add an mp3 file to the app's assets, the following code will add the mp3 to the media player library:
private void AddSong()
{
Uri file = new Uri("Assets/someSong.mp3", UriKind.Relative);
//copy file to isolated storage
var myIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication();
var fileStream = myIsolatedStorage.CreateFile("someSong.mp3");
var resource = Application.GetResourceStream(file);
int chunkSize = 4096;
byte[] bytes = new byte[chunkSize];
int byteCount;
while ((byteCount = resource.Stream.Read(bytes, 0, chunkSize)) > 0)
{
fileStream.Write(bytes, 0, byteCount);
}
fileStream.Close();
Microsoft.Xna.Framework.Media.PhoneExtensions.SongMetadata metaData = new Microsoft.Xna.Framework.Media.PhoneExtensions.SongMetadata();
metaData.AlbumName = "Some Album name";
metaData.ArtistName = "Some Artist Name";
metaData.GenreName = "test";
metaData.Name = "someSongName";
var ml = new MediaLibrary();
Uri songUri = new Uri("someSong.mp3", UriKind.RelativeOrAbsolute);
var song = Microsoft.Xna.Framework.Media.PhoneExtensions.MediaLibraryExtensions.SaveSong(ml, songUri, metaData, Microsoft.Xna.Framework.Media.PhoneExtensions.SaveSongOperation.CopyToLibrary);
}
I also needed to add:
using System.IO.IsolatedStorage;
I would love to claim credit for this, but I found the answer here:
http://social.msdn.microsoft.com/forums/wpapps/en-US/f5fa73da-176b-4aaa-8960-8f704236bda5/medialibrary-savesong-method

By default the media library on the emulator is empty. I also do not think it is possible to automagically hook up your dev machine's music folder to the emulator to test that way. It might be possible to manually configure the emulated phone with an email account! and save music onto it that way, but even if that worked you'd have to do it each and every time you restart the emulator.
Best way to test would be t deploy to a real device.