EmguCV Mat object is always null in .NET MAUI Android - c#

I want to capture camera frames by using Emgu.CV in .NET MAUI and I have Emgu.CV runtime for MAUI Android. I can access the camera but I couldn't transfer the feed to Mat object. It's always null. The same code is working on Windows Forms and WPF but on MAUI it's not. Also _capture.ImageGrabbed += _capture_ImageGrabbed; doesn't trigger _capture_ImageGrabbed event.
AndroidManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<application android:allowBackup="true" android:icon="#mipmap/appicon" android:roundIcon="#mipmap/appicon_round" android:supportsRtl="true"></application>
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
<uses-permission android:name="android.permission.READ_MEDIA_AUDIO" />
<uses-permission android:name="android.permission.READ_MEDIA_VIDEO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_VIDEO" />
<queries>
<intent>
<action android:name="android.media.action.IMAGE_CAPTURE" />
</intent>
<intent>
<action android:name="android.media.action.VIDEO_CAPTURE" />
</intent>
</queries>
</manifest>
MainPage.xaml.cs
public partial class MainPage : ContentPage
{
private VideoCapture _capture = null;
//private bool _captureInProgress;
private Mat _frame;
public MainPage()
{
InitializeComponent();
//CvInvoke.UseOpenCL = false;
CvInvokeAndroid.Init();
_capture = new VideoCapture(0, VideoCapture.API.Android);
_capture.ImageGrabbed += _capture_ImageGrabbed;
_frame = new Mat();
}
private void captureButton_Clicked(object sender, EventArgs e)
{
_capture.Start();
}
private void _capture_ImageGrabbed(object sender, EventArgs e)
{
_capture.Retrieve(_frame, 0);
}
private void ManuelFrame_Clicked(object sender, EventArgs e)
{
_capture.Retrieve(_frame, 0);
if (_frame.IsEmpty)
{
statusLabel.Text = "frame is null";
}
else statusLabel.Text = "frame is not null";
}
}
Images
As you can see, After Retrieve method, Mat object shouldn't be null.
I can access the camera with capture but I can't transfer the capture to _frame object. I tried to change Android API to 33 (Android 13) it didn't work. I put a manuel event for retrieving frame it also didn't work. What can I do?
UPDATE
I implemented the code which is released for MAUI from EmguCV Github page.
However, the ImageGrabbed event is never triggered and Mat objects are still empty.
GitHub Implementation
public partial class MainPage : ContentPage
{
private VideoCapture _capture = null;
//private bool _captureInProgress;
private Mat _mat;
private Mat _renderMat;
private string _defaultButtonText = "Start Camera";
protected string _StopCameraButtonText = "Stop Camera";
private string _deaultImage;
ButtonTextImagePage buttonPage = new ButtonTextImagePage();
private IProcessAndRenderModel _model;
public MainPage()
{
InitializeComponent();
//CvInvoke.UseOpenCL = false;
CvInvokeAndroid.Init();
InitVideoCapture();
}
private void _capture_ImageGrabbed(object sender, EventArgs e)
{
if (_mat == null)
_mat = new Mat();
_capture.Retrieve(_mat);
if (_renderMat == null)
_renderMat = new Mat();
_mat.CopyTo(_renderMat);
}
protected virtual async void captureButton_Clicked(object sender, EventArgs e)
{
var button = buttonPage.GetButton();
if(captureButton.Text.Equals(_StopCameraButtonText))
{
#if __ANDROID__ && __USE_ANDROID_CAMERA2__
StopCapture();
//AndroidImageView.Visibility = ViewStates.Invisible;
#else
_capture.Stop();
_capture.Dispose();
_capture = null;
#endif
captureButton.Text = _defaultButtonText;
return;
}
Mat[] images;
if(_deaultImage == null)
{
images = new Mat[0];
}
else
{
images = await buttonPage.LoadImages(new string[] { _deaultImage });
if (images == null || (images.Length > 0 && images[0] == null)) return;
}
if(images.Length == 0)
{
#if __ANDROID__ && __USE_ANDROID_CAMERA2__
StartCapture(async delegate (Object captureSender, Mat m)
{
//Skip the frame if busy,
//Otherwise too many frames arriving and will eventually saturated the memory.
if (!_isBusy)
{
_isBusy = true;
try
{
String message = String.Empty;
await Task.Run(() =>
{
if (_renderMat == null)
_renderMat = new Mat();
using (InputArray iaImage = m.GetInputArray())
{
iaImage.CopyTo(_renderMat);
}
message = _model.ProcessAndRender(m, _renderMat);
});
SetImage(_renderMat);
SetMessage(message);
}
finally
{
_isBusy = false;
}
}
});
#else
//Handle video
if (_capture == null)
{
InitVideoCapture();
}
if (_capture != null)
_capture.Start();
#endif
captureButton.Text = _StopCameraButtonText;
}
else
{
if (_renderMat == null) _renderMat = new Mat();
images[0].CopyTo(_renderMat);
}
}
private void InitVideoCapture()
{
var openCVConfigDict = CvInvoke.ConfigDict;
bool haveVideoio = (openCVConfigDict["HAVE_OPENCV_VIDEOIO"] != 0);
if (haveVideoio && Microsoft.Maui.Devices.DeviceInfo.Platform == DevicePlatform.Android)
{
#if __ANDROID__ && __USE_ANDROID_CAMERA2__
return true;
#else
if (CvInvoke.Backends.Length > 0)
{
if (Microsoft.Maui.Devices.DeviceInfo.Platform == DevicePlatform.Android)
{
_capture = new VideoCapture(0, VideoCapture.API.Android);
}
else
{
_capture = new VideoCapture();
}
if (_capture.IsOpened)
{
_capture.ImageGrabbed += _capture_ImageGrabbed;
return;
}
else
{
_capture.Dispose();
_capture = null;
}
}
#endif
}
return;
}

I found the problem. Android Emulator is not working correctly with EmguCV on .NET MAUI. I deployed the application my physical device and it worked. However, OpenCV's video capture is broken for android. I opened the camera but all time image is in black-gray format. If you working with real-time camera you can make two things. Install android studio and change the camera source with your webcam or directly build on your physical device.

Related

How to remove video black bands in videoView with LibVlcSharp?

when I play a video with video, it presents black bands in the top and bottom part of the video, like in image in the URL
https://imgur.com/a/JiUv8rt. I'd like to remove the bands and display just the video in an absolute layout. how can I reach my goal?
<pages:PopupPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
xmlns:d="http://xamarin.com/schemas/2014/forms/design"
xmlns:pages="clr-namespace:Rg.Plugins.Popup.Pages;assembly=Rg.Plugins.Popup"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d"
xmlns:animations="clr-namespace:Rg.Plugins.Popup.Animations;assembly=Rg.Plugins.Popup"
xmlns:shared="clr-namespace:LibVLCSharp.Forms.Shared;assembly=LibVLCSharp.Forms"
x:Class="App.Pages.WebcamVideoPopUpPage"
BackgroundColor="Transparent">
<pages:PopupPage.Animation>
<animations:ScaleAnimation
PositionIn="Center"
PositionOut="Center"
ScaleIn="1.2"
ScaleOut="0.8"
DurationIn="400"
DurationOut="300"
EasingIn="SinOut"
EasingOut="SinIn"
HasBackgroundAnimation="True"/>
</pages:PopupPage.Animation>
<AbsoluteLayout x:Name="AbsoluteLayoutWebcam"
HorizontalOptions="FillAndExpand"
VerticalOptions="FillAndExpand">
<shared:VideoView x:Name="VideoViewWebcam"
AbsoluteLayout.LayoutFlags="All"
AbsoluteLayout.LayoutBounds="0, 0, 1, 1"
MediaPlayer ="{Binding MediaPlayer}"
MediaPlayerChanged ="VideoView_MediaPlayerChanged"/>
<Label x:Name="DescriptionWebcam"
AbsoluteLayout.LayoutFlags="All"
AbsoluteLayout.LayoutBounds="0, 0, 1, .2"
HorizontalOptions="Fill"
VerticalOptions="Fill"
Text="{Binding InfoWebcam}"
FontSize="Large"
TextColor="White"/>
</AbsoluteLayout>
</pages:PopupPage>
UPDATE
I update at latest pre-release as #mtz suggested me, and I modified my code in the following way:
public partial class WebcamVideoPopUpPage : PopupPage
{
public WebcamVideoPopUpPage()
{
var vm = App.Locator.WebCamVideoVM;
this.BindingContext = vm;
InitializeComponent();
MediaPlayerWebcam.VideoView.MediaPlayerChanged += VideoView_MediaPlayerChanged;
MediaPlayerWebcam.MediaPlayer.AspectRatio = "FitScreen";
}
protected override void OnAppearing()
{
base.OnAppearing();
Messenger.Default.Send(new OnApperingVideoMessage());
}
private void VideoView_MediaPlayerChanged(object sender, LibVLCSharp.Shared.MediaPlayerChangedEventArgs e)
{
Messenger.Default.Send(new OnVideoViewInitializedMessage());
}
protected override void OnDisappearing()
{
base.OnDisappearing();
MediaPlayerWebcam.MediaPlayer.Stop();
MediaPlayerWebcam.MediaPlayer = null;
}
}
My xaml:
<AbsoluteLayout x:Name="AbsoluteLayoutWebcam"
HorizontalOptions="FillAndExpand"
VerticalOptions="FillAndExpand">
<shared:MediaPlayerElement x:Name="MediaPlayerWebcam"
AbsoluteLayout.LayoutFlags="All"
AbsoluteLayout.LayoutBounds="0, 0, 1, .4"
MediaPlayer ="{Binding MediaPlayer}"/>
<Label x:Name="DescriptionWebcam"
AbsoluteLayout.LayoutFlags="All"
AbsoluteLayout.LayoutBounds="0, 0, 1, .2"
HorizontalOptions="Fill"
VerticalOptions="Fill"
Text="{Binding InfoWebcam}"
FontSize="Large"
TextColor="White"/>
</AbsoluteLayout>
My viewModel:
public class WebcamVideoViewModel : BaseViewModel
{
private LibVLC LibVLC { get; set; }
private bool IsLoaded { get; set; }
private bool IsVideoViewInitialized { get; set; }
private Media Media { get; set; }
private MediaPlayer _mediaPlayer;
public MediaPlayer MediaPlayer
{
get { return _mediaPlayer; }
set
{
_mediaPlayer = value;
OnPropertyChanged();
}
}
private string _InfoWebcam { get; set; }
public string InfoWebcam
{
get { return _InfoWebcam; }
set
{
_InfoWebcam = value;
OnPropertyChanged();
}
}
public WebcamVideoViewModel(INavigationService navigationService, IApiAutostradeManagerFactory apiAutostradeFactory) : base(navigationService, apiAutostradeFactory)
{
Messenger.Default.Register<InfoWebcamVideoMessage>(this, OnReceivedInfoWebcam);
Messenger.Default.Register<OnApperingVideoMessage>(this, OnAppearing);
Messenger.Default.Register<OnVideoViewInitializedMessage>(this, OnVideoViewInitialized);
Task.Run(Initialize);
}
private void Initialize()
{
Core.Initialize();
LibVLC = new LibVLC();
MediaPlayer = new MediaPlayer(LibVLC);
}
private async void OnReceivedInfoWebcam(InfoWebcamVideoMessage msg)
{
var response = await ApiManager.GetVideoWebcam(msg.Mpr, msg.Uuid);
if (response.IsSuccessStatusCode)
{
InfoWebcam = msg.T_Description_webcam;
var stream = await response.Content.ReadAsStreamAsync();
Media = new Media(LibVLC, stream);
Play();
}
}
public void OnAppearing(OnApperingVideoMessage msg)
{
IsLoaded = true;
}
public void OnVideoViewInitialized(OnVideoViewInitializedMessage msg)
{
IsVideoViewInitialized = true;
}
private void Play()
{
if (IsLoaded && IsVideoViewInitialized)
{
MediaPlayer.Play(Media);
}
}
}
Now i'm closer to solution because videoView is resizable with the button present at bootm, but I'd like to get at start the fill AspectRatio and I don't want anything expect the video(In other words, I'd want to remove seek bar and resize video button). Another problem is that after I close mediaPlayer, and I open again a new video, my app crashes. Any advice?
You need to change the aspect ratio to "fill the screen".
See how to here: https://github.com/videolan/libvlcsharp/blob/91b8f06ee1bedd9b3219a4e9ade0a9c44f59fda8/LibVLCSharp.Forms/Shared/PlaybackControls.xaml.cs#L926 or use the latest pre-release LibVLCSharp.Forms package that contains the MediaPlayerElement which has this feature built-in (soon in stable version).
What you just trying is to stretch the video. But remember it will effect video quality.
To remain simple, you can try this working and tested code:
MediaPlayerWebcam.MediaPlayer.AspectRatio = $"{MediaPlayerWebcam.Width.ToString()}:{MediaPlayerWebcam.Height.ToString()}";
MediaPlayerWebcam.Scale = 0;
My scenario is a fullscreen player, I do it with these codes refer to LibVLCSharp.Forms, hope it will be helpful. the code deal with fullscreen(commented) or fill screen with video.
public void PlayerFullScreen()
{
//restore
if (_isFullScreen)
{
RestoreDefaultWindowInfo();
_isFullScreen = false;
_mediaPlayer.Scale = _originalScale; //reset
_mediaPlayer.AspectRatio = _originalAspectRatio;
//Mouse.Capture(null);
playerBar.Visibility = Visibility.Visible;
}
else // fullscreen(stretch video)
{
this.WindowStyle = WindowStyle.None;
this.ResizeMode = ResizeMode.NoResize;
this.Left = 0;
this.Top = 0;
this.Width = SystemParameters.VirtualScreenWidth;
this.Height = SystemParameters.VirtualScreenHeight;
//this.Topmost = true;
_isFullScreen = true;
_originalScale = _mediaPlayer.Scale; // save original
_originalAspectRatio = _mediaPlayer.AspectRatio;
playerBar.Visibility = Visibility.Collapsed;
MediaTrack? mediaTrack;
try
{
mediaTrack = _mediaPlayer.Media?.Tracks?.FirstOrDefault(x => x.TrackType == TrackType.Video);
}
catch (Exception)
{
mediaTrack = null;
}
if (mediaTrack == null || !mediaTrack.HasValue)
{
return;
}
//get windows scale factor(DPI)
PresentationSource source = PresentationSource.FromVisual(this);
double dpiX=1.0, dpiY=1.0;
if (source != null)
{
dpiX = source.CompositionTarget.TransformToDevice.M11;
dpiY = source.CompositionTarget.TransformToDevice.M22;
}
var displayW = this.Width * dpiX;
var displayH = this.Height * dpiY;
var videoSwapped = mediaTrack.Value.Data.Video.Orientation == VideoOrientation.LeftBottom ||
mediaTrack.Value.Data.Video.Orientation == VideoOrientation.RightTop;
var videoW = mediaTrack.Value.Data.Video.Width;
var videoH = mediaTrack.Value.Data.Video.Height;
if (videoSwapped)
{
var swap = videoW;
videoW = videoH;
videoH = swap;
}
if (mediaTrack.Value.Data.Video.SarNum != mediaTrack.Value.Data.Video.SarDen)
videoW = videoW * mediaTrack.Value.Data.Video.SarNum / mediaTrack.Value.Data.Video.SarDen;
var ar = videoW / (float)videoH;
var dar = displayW / (float)displayH;
float scale;
if (dar >= ar)
scale = (float)displayW / videoW; /* horizontal */
else
scale = (float)displayH / videoH; /* vertical */
//keep ratio of width/height, not fill the srceen
//_mediaPlayer.Scale = scale; // 这是保持视频宽高比的,视频不会变形
//_mediaPlayer.AspectRatio = string.Empty;//这是保持视频宽高比的,视频不会变形
//这是视频变形适配屏幕的情况(满屏)
//fill all the screen by video,video is streched
float xscale, yscale;
xscale = (float)(displayW / videoW);
yscale = (float)(displayH / videoH);
_mediaPlayer.Scale = (xscale<yscale)? xscale:yscale;
string aspectRatio = String.Format("{0}:{1}",
this.Width,this.Height);
_mediaPlayer.AspectRatio = aspectRatio;
}
}

Turn camera picture depending on the device orientation (Xamarin.iOS)

I want to make a Xamarin.iOS app where I can capture pictures like the camera... My app supports only portrait.
I want to turn the captured picture to portrait if the camera was landscape when i capture the picture.
Does someone know how I can do this?
Code
public async void CapturePhoto()
{
var videoConnection = stillImageOutput.ConnectionFromMediaType(AVMediaType.Video);
var sampleBuffer = await stillImageOutput.CaptureStillImageTaskAsync(videoConnection);
var jpegImageAsBytes = AVCaptureStillImageOutput.JpegStillToNSData(sampleBuffer).ToArray();
string base64StringImage = Convert.ToBase64String(jpegImageAsBytes);
FaceRecognition faceRecognition = new FaceRecognition();
int result = faceRecognition.SendPhoto(base64StringImage);
}
Try this :
var currentOrientation = UIApplication.SharedApplication.StatusBarOrientation;
if (currentOrientation == UIInterfaceOrientation.Portrait)
{
videoConnection.VideoOrientation = AVCaptureVideoOrientation.Portrait;
}
else if (currentOrientation == UIInterfaceOrientation.LandscapeRight)
{
videoConnection.VideoOrientation = AVCaptureVideoOrientation.LandscapeRight;
}
//xxx
Update:
If the app only supports an orientation or you lock the screen , there is another old way to detect device orientation. Core Motion
public void LockOrientation()
{
CMMotionManager CMManager = new CMMotionManager();
CMManager.DeviceMotionUpdateInterval = 0.2f;
CMManager.StartDeviceMotionUpdates(NSOperationQueue.MainQueue, (motion, error) => {
if (Math.Abs(motion.Gravity.X) > Math.Abs(motion.Gravity.Y))
{
Console.WriteLine("Lan");
if (motion.Gravity.X > 0)
{
UIDevice.CurrentDevice.SetValueForKey(new NSNumber((int)UIInterfaceOrientation.LandscapeLeft), new NSString("orientation"));
Console.WriteLine("Left");
}
else
{
UIDevice.CurrentDevice.SetValueForKey(new NSNumber((int)UIInterfaceOrientation.LandscapeRight), new NSString("orientation"));
Console.WriteLine("Right");
}
}
else
{
if (motion.Gravity.Y >= 0)
{
Console.WriteLine("Down");
}
else
{
Console.WriteLine("UP");
}
}
CMManager.StopDeviceMotionUpdates();
});
}
Refer to here

How properly play an audio file to SKYPE4COM

I'm developing the skype plugin. It should play an audio file during the call.
Audio player use naudio library
public class AudioPlayback : IDisposable, IPlayer
{
WaveStream _outStream;
IWavePlayer _player;
IWaveIn _recorder;
public event Action<byte[], int> DataAvailable;
public AudioPlayback()
{
_recorder = new WaveInEvent();
_recorder.WaveFormat = new WaveFormat(16000, 16, 1);
_recorder.DataAvailable += OnRecorderDataAvailable;
}
private void OnRecorderDataAvailable(object sender, WaveInEventArgs e)
{
if (DataAvailable!= null)
{
DataAvailable(e.Buffer, e.BytesRecorded);
}
}
public void LoadFile(string fileName)
{
_outStream = new Mp3FileReader(fileName);
if (_outStream.WaveFormat.Encoding != WaveFormatEncoding.Pcm)
{
_outStream = WaveFormatConversionStream.CreatePcmStream(_outStream);
}
}
private void CreatePlayer()
{
if (_player == null)
{
var waveOut = new WaveOut();
waveOut.DesiredLatency = 200;
waveOut.NumberOfBuffers = 2;
waveOut.DeviceNumber = 0;
_player = waveOut;
}
}
public void Play()
{
CreatePlayer();
if (_player.PlaybackState != PlaybackState.Playing)
{
if (_player.PlaybackState == PlaybackState.Stopped)
_recorder.StartRecording();
_player.Init(_outStream);
_player.Play();
}
}
}
In skype plugin class I create NetworkStream and TcpListener. Use event from player to get buffer data and write to the network stream
WriteToStream(buffer, 0, num);
On call started I change input for skype
call.InputDevice[TCallIoDeviceType.callIoDeviceTypeSoundcard] = "";
call.InputDevice[TCallIoDeviceType.callIoDeviceTypePort] = _inputPort.ToString();
Was fighting with this for several hours. Finally got the sound on the another skype only when turn on Stereo Mixer (Recording devices).
The question: is this the right way? I don't like a bit to play sound and capture it. But here I have a positive thing - I capture exactly with parameters appropriate for skype.

how to set capture source of AudioVideoCaptureDevice on Windows Phone

I'm trying to capture video on Windows Phone using the AudioVideoCaptureDevice but when I try to set the captureSource I get a error message saying " Operation not valid due to the state of the object". Can you tell me what to do right in the following code?
Code snippet:
// Viewfinder for capturing video.
private VideoBrush videoRecorderBrush;
// Source and device for capturing video.
private CaptureSource _cs;
private VideoCaptureDevice _cd;
private AudioVideoCaptureDevice vcDevice;
double w, h;
// File details for storing the recording.
private IsolatedStorageFileStream isoVideoFile;
private FileSink fileSink;
private string isoVideoFileName = "iClips_Video.mp4";
private StorageFile sfVideoFile;
// For managing button and application state.
private enum ButtonState { Initialized, Stopped, Ready, Recording, Playback, Paused, NoChange, CameraNotSupported };
private ButtonState currentAppState;
// Constructor
public MainPage()
{
try
{
InitializeComponent();
//setup recording
// Prepare ApplicationBar and buttons.
PhoneAppBar = (ApplicationBar)ApplicationBar;
PhoneAppBar.IsVisible = true;
StartRecording = ((ApplicationBarIconButton)ApplicationBar.Buttons[0]);
StopPlaybackRecording = ((ApplicationBarIconButton)ApplicationBar.Buttons[1]);
StartPlayback = ((ApplicationBarIconButton)ApplicationBar.Buttons[2]);
PausePlayback = ((ApplicationBarIconButton)ApplicationBar.Buttons[3]);
SetScreenResolution();
//initialize the camera task
cameraCaptureTask = new CameraCaptureTask();
cameraCaptureTask.Completed += new EventHandler<PhotoResult>(cameraCaptureTask_Completed);
if (isFilePresent("username") && isFilePresent("Password"))
{
if (isFilePresent("IsProfilePhotoOnServer"))
{
connectToSocket();
}
else
{
SignUpProfilePhoto();
}
}
else
{
SignIn();
}
}
catch (Exception ex)
{
this.Dispatcher.BeginInvoke(delegate()
{
MessageBox.Show("Constructor Error:\n"+ ex.Message);
});
}
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
// Initialize the video recorder.
InitializeVideoRecorder();
//prepare shutter hot keys
CameraButtons.ShutterKeyHalfPressed += OnButtonHalfPress;
CameraButtons.ShutterKeyPressed += OnButtonFullPress;
CameraButtons.ShutterKeyReleased += OnButtonRelease;
}
protected override void OnNavigatedFrom(NavigationEventArgs e)
{
// Dispose of camera and media objects.
DisposeVideoPlayer();
DisposeVideoRecorder();
base.OnNavigatedFrom(e);
CameraButtons.ShutterKeyHalfPressed -= OnButtonHalfPress;
CameraButtons.ShutterKeyPressed -= OnButtonFullPress;
CameraButtons.ShutterKeyReleased -= OnButtonRelease;
}
protected override void OnOrientationChanged(OrientationChangedEventArgs e)
{
if (vcDevice != null)
{
if (e.Orientation == PageOrientation.LandscapeLeft)
{
txtDebug.Text = "LandscapeLeft";
videoRecorderBrush.RelativeTransform =
new CompositeTransform() { CenterX = 0.5, CenterY = 0.5, Rotation = 90 };
//rotate logo
if (logo != null)
{
RotateTransform rt = new RotateTransform();
rt.Angle = 90;
//default rotation is around top left corner of the control,
//but you sometimes want to rotate around the center of the control
//to do that, you need to set the RenderTransFormOrigin
//of the item you're going to rotate
//I did not test this approach, maybe You're going to need to use actual coordinates
//so this bit is for information purposes only
logo.RenderTransformOrigin = new Point(0.5, 0.5);
logo.RenderTransform = rt;
}
//rotate sign in link
if (MyGrid != null)
{
RotateTransform rt = new RotateTransform();
rt.Angle = 90;
//default rotation is around top left corner of the control,
//but you sometimes want to rotate around the center of the control
//to do that, you need to set the RenderTransFormOrigin
//of the item you're going to rotate
//I did not test this approach, maybe You're going to need to use actual coordinates
//so this bit is for information purposes only
MyGrid.RenderTransformOrigin = new Point(0.5, 0.5);
MyGrid.RenderTransform = rt;
}
}
if (e.Orientation == PageOrientation.PortraitUp)
{
txtDebug.Text = "PortraitUp";
videoRecorderBrush.RelativeTransform =
new CompositeTransform() { CenterX = 0.5, CenterY = 0.5, Rotation = 90 };
//rotate logo
if (logo != null)
{
RotateTransform rt = new RotateTransform();
rt.Angle = 0;
//default rotation is around top left corner of the control,
//but you sometimes want to rotate around the center of the control
//to do that, you need to set the RenderTransFormOrigin
//of the item you're going to rotate
//I did not test this approach, maybe You're going to need to use actual coordinates
//so this bit is for information purposes only
logo.RenderTransformOrigin = new Point(0.5, 0.5);
logo.RenderTransform = rt;
}
//rotate sign in link
if (MyGrid != null)
{
RotateTransform rt = new RotateTransform();
rt.Angle = 0;
//default rotation is around top left corner of the control,
//but you sometimes want to rotate around the center of the control
//to do that, you need to set the RenderTransFormOrigin
//of the item you're going to rotate
//I did not test this approach, maybe You're going to need to use actual coordinates
//so this bit is for information purposes only
MyGrid.RenderTransformOrigin = new Point(0.5, 0.5);
MyGrid.RenderTransform = rt;
}
}
if (e.Orientation == PageOrientation.LandscapeRight)
{
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = "LandscapeRight";
// Rotate for LandscapeRight orientation.
//videoRecorderBrush.RelativeTransform =
//new CompositeTransform() { CenterX = 0.5, CenterY = 0.5, Rotation = 180 };
//rotate logo
if (logo != null)
{
RotateTransform rt = new RotateTransform();
rt.Angle = -90;
//default rotation is around top left corner of the control,
//but you sometimes want to rotate around the center of the control
//to do that, you need to set the RenderTransFormOrigin
//of the item you're going to rotate
//I did not test this approach, maybe You're going to need to use actual coordinates
//so this bit is for information purposes only
logo.RenderTransformOrigin = new Point(0.5, 0.5);
logo.RenderTransform = rt;
}
//rotate MyGrid
if (MyGrid != null)
{
RotateTransform rt = new RotateTransform();
rt.Angle = -90;
//default rotation is around top left corner of the control,
//but you sometimes want to rotate around the center of the control
//to do that, you need to set the RenderTransFormOrigin
//of the item you're going to rotate
//I did not test this approach, maybe You're going to need to use actual coordinates
//so this bit is for information purposes only
MyGrid.RenderTransformOrigin = new Point(0.5, 0.5);
MyGrid.RenderTransform = rt;
}
});
}
if (e.Orientation == PageOrientation.PortraitDown)
{
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = "PortraitDown";
videoRecorderBrush.RelativeTransform =
new CompositeTransform() { CenterX = 0.5, CenterY = 0.5, Rotation = 270 };
});
}
}
}
// Hardware shutter button Hot-Actions.
private void OnButtonHalfPress(object sender, EventArgs e)
{
//toggle between video- play and pause
try
{
this.Dispatcher.BeginInvoke(delegate()
{
if (StartPlayback.IsEnabled)
{
PlayVideo();
}
if (PausePlayback.IsEnabled)
{
PauseVideo();
}
});
}
catch (Exception focusError)
{
// Cannot focus when a capture is in progress.
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = focusError.Message;
});
}
}
private void OnButtonFullPress(object sender, EventArgs e)
{
// Focus when a capture is not in progress.
try
{
this.Dispatcher.BeginInvoke(delegate()
{
if (vcDevice != null)
{
//stopVideoPlayer if it's playing back
if (currentAppState == ButtonState.Playback || currentAppState == ButtonState.Paused)
{
DisposeVideoPlayer();
StartVideoPreview();
}
if (StartRecording.IsEnabled)
{
StartVideoRecording();
}
else
{
StopVideoRecording();
}
}
});
}
catch (Exception focusError)
{
// Cannot focus when a capture is in progress.
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = focusError.Message;
});
}
}
private void OnButtonRelease(object sender, EventArgs e)
{
try
{
this.Dispatcher.BeginInvoke(delegate()
{
});
}
catch (Exception focusError)
{
// Cannot focus when a capture is in progress.
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = focusError.Message;
});
}
}
// Update the buttons and text on the UI thread based on app state.
private void UpdateUI(ButtonState currentButtonState, string statusMessage)
{
// Run code on the UI thread.
Dispatcher.BeginInvoke(delegate
{
switch (currentButtonState)
{
// When the camera is not supported by the phone.
case ButtonState.CameraNotSupported:
StartRecording.IsEnabled = false;
StopPlaybackRecording.IsEnabled = false;
StartPlayback.IsEnabled = false;
PausePlayback.IsEnabled = false;
break;
// First launch of the application, so no video is available.
case ButtonState.Initialized:
StartRecording.IsEnabled = true;
StopPlaybackRecording.IsEnabled = false;
StartPlayback.IsEnabled = false;
PausePlayback.IsEnabled = false;
break;
// Ready to record, so video is available for viewing.
case ButtonState.Ready:
StartRecording.IsEnabled = true;
StopPlaybackRecording.IsEnabled = false;
StartPlayback.IsEnabled = true;
PausePlayback.IsEnabled = false;
break;
// Video recording is in progress.
case ButtonState.Recording:
StartRecording.IsEnabled = false;
StopPlaybackRecording.IsEnabled = true;
StartPlayback.IsEnabled = false;
PausePlayback.IsEnabled = false;
break;
// Video playback is in progress.
case ButtonState.Playback:
StartRecording.IsEnabled = false;
StopPlaybackRecording.IsEnabled = true;
StartPlayback.IsEnabled = false;
PausePlayback.IsEnabled = true;
break;
// Video playback has been paused.
case ButtonState.Paused:
StartRecording.IsEnabled = false;
StopPlaybackRecording.IsEnabled = true;
StartPlayback.IsEnabled = true;
PausePlayback.IsEnabled = false;
break;
default:
break;
}
// Display a message.
txtDebug.Text = statusMessage;
// Note the current application state.
currentAppState = currentButtonState;
});
}
public async void InitializeVideoRecorder()
{
try
{
if (_cs == null)
{
_cs = new CaptureSource();
fileSink = new FileSink();
_cd = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
CameraSensorLocation location = CameraSensorLocation.Back;
var captureResolutions =
AudioVideoCaptureDevice.GetAvailableCaptureResolutions(location);
vcDevice = await AudioVideoCaptureDevice.OpenAsync(location, captureResolutions[0]);
vcDevice.RecordingFailed += OnCaptureFailed;
vcDevice.VideoEncodingFormat = CameraCaptureVideoFormat.H264;
vcDevice.AudioEncodingFormat = CameraCaptureAudioFormat.Aac;
// Initialize the camera if it exists on the phone.
if (vcDevice != null)
{
//initialize fileSink
await InitializeFileSink();
// Create the VideoBrush for the viewfinder.
videoRecorderBrush = new VideoBrush();
videoRecorderBrush.SetSource(_cs);
// Display the viewfinder image on the rectangle.
viewfinderRectangle.Fill = videoRecorderBrush;
_cs.Start();
// Set the button state and the message.
UpdateUI(ButtonState.Initialized, "Tap record to start recording...");
}
else
{
// Disable buttons when the camera is not supported by the phone.
UpdateUI(ButtonState.CameraNotSupported, "A camera is not supported on this phone.");
}
}
}
catch(Exception ex)
{
MessageBox.Show("InitializeVideoRecorder Error:\n" + ex.Message);
}
}
public async Task InitializeFileSink()
{
StorageFolder isoStore = ApplicationData.Current.LocalFolder;
sfVideoFile = await isoStore.CreateFileAsync(
isoVideoFileName,
CreationCollisionOption.ReplaceExisting);
}
private void OnCaptureFailed(AudioVideoCaptureDevice sender, CaptureFailedEventArgs args)
{
MessageBox.Show(args.ToString());
}
private void OnCaptureSourceFailed(object sender, ExceptionRoutedEventArgs e)
{
MessageBox.Show(e.ErrorException.Message.ToString());
}
// Set the recording state: display the video on the viewfinder.
private void StartVideoPreview()
{
try
{
// Display the video on the viewfinder.
if (_cs.VideoCaptureDevice != null
&& _cs.State == CaptureState.Stopped)
{
// Add captureSource to videoBrush.
videoRecorderBrush.SetSource(_cs);
// Add videoBrush to the visual tree.
viewfinderRectangle.Fill = videoRecorderBrush;
_cs.Start();
// Set the button states and the message.
UpdateUI(ButtonState.Ready, "Ready to record.");
//Create optional Resolutions
}
}
// If preview fails, display an error.
catch (Exception e)
{
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = "ERROR: " + e.Message.ToString();
});
}
}
// Set recording state: start recording.
private void StartVideoRecording()
{
try
{
// Connect fileSink to captureSource.
if (_cs.VideoCaptureDevice != null
&& _cs.State == CaptureState.Started)
{
_cs.Stop();
// Connect the input and output of fileSink.
fileSink.CaptureSource = _cs;
fileSink.IsolatedStorageFileName = isoVideoFileName;
}
// Begin recording.
if (_cs.VideoCaptureDevice != null
&& _cs.State == CaptureState.Stopped)
{
_cs.Start();
}
// Set the button states and the message.
UpdateUI(ButtonState.Recording, "Recording...");
StartTimer();
}
// If recording fails, display an error.
catch (Exception e)
{
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = "ERROR: " + e.Message.ToString();
});
}
}
// Set the recording state: stop recording.
private void StopVideoRecording()
{
try
{
// Stop recording.
if (_cs.VideoCaptureDevice != null
&& _cs.State == CaptureState.Started)
{
_cs.Stop();
// Disconnect fileSink.
fileSink.CaptureSource = null;
fileSink.IsolatedStorageFileName = null;
// Set the button states and the message.
UpdateUI(ButtonState.Stopped, "Preparing viewfinder...");
StopTimer();
StartVideoPreview();
}
}
// If stop fails, display an error.
catch (Exception e)
{
this.Dispatcher.BeginInvoke(delegate()
{
txtDebug.Text = "ERROR: " + e.Message.ToString();
});
}
}
// Start the video recording.
private void StartRecording_Click(object sender, EventArgs e)
{
// Avoid duplicate taps.
StartRecording.IsEnabled = false;
StartVideoRecording();
}
private void DisposeVideoRecorder()
{
if (_cs != null)
{
// Stop captureSource if it is running.
if (_cs.VideoCaptureDevice != null
&& _cs.State == CaptureState.Started)
{
_cs.Stop();
}
// Remove the event handler for captureSource.
_cs.CaptureFailed -= OnCaptureFailed;
// Remove the video recording objects.
_cs = null;
vcDevice = null;
fileSink = null;
videoRecorderBrush = null;
}
}
private void OnCaptureFailed(object sender, ExceptionRoutedEventArgs e)
{
throw new NotImplementedException();
}
//STOP WATCH
private void StartTimer()
{
dispatcherTimer = new System.Windows.Threading.DispatcherTimer();
dispatcherTimer.Tick += new EventHandler(dispatcherTimer_Tick);
dispatcherTimer.Interval = new TimeSpan(0, 0, 1);
dispatcherTimer.Start();
startTime = System.DateTime.Now;
}
private void StopTimer()
{
dispatcherTimer.Stop();
}
private void dispatcherTimer_Tick(object sender, EventArgs e)
{
System.DateTime now = System.DateTime.Now;
txtRecTime.Text = now.Subtract(startTime).ToString();
}
the error is thrown inside initializeVideoRecorder().
Seems like you're code is too long to analyze as #john mentioned in the comment. But since you've suggested to post some helpful links here you go: Following these steps here in msdn, would work without any issues as I've tried it once.
If you really need a sample to kick off, there's one from msdn. Hope you'll maximize the utilities.
I figured out I have to pass the capture device to the videobrush to make it work. I'm still not sure where and if I should use CaptureSource when capturing with AudioVideoCaptureDevice. Anyway here is the solution code:
// Create the VideoBrush for the viewfinder.
videoRecorderBrush = new VideoBrush();
videoRecorderBrush.SetSource(vcDevice); //substitute vcDevice with captureSource - vcDevice is the reference of AudioVideoCaptureDevice
// Display the viewfinder image on the rectangle.
viewfinderRectangle.Fill = videoRecorderBrush;

using Nito AsyncEx/AsyncInLine to lead with async filePicker

I'm new at async world and i started developing some stuff for windows store apps.
My problem is that i need to wait the async method of filepicker ends to do other stuff.
So i tried search for: "run synchronous and asynchronous method c#" and i ended on Stephen Cleary blog. I tryed to use Nito AsyncEx to lead with my problem and it doesn't work because when i call the following method it doesn't stop.
private List<Templates> classModel = new List<Templates>();
private void btnLoadClassExercises_Click(object sender, RoutedEventArgs e)
{
AsyncContext.Run(() => loadFile());
// do some stuff with classModel
}
private async void loadFile()
{
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.FileTypeFilter.Add(".xml");
var file = await openPicker.PickSingleFileAsync();
classModel = xml.getControlClass(file);
}
During my search i found a classe called AsyncInLine that helped me out with some others stuffs like this one. But with this specific situation, i didn't work too. (http://social.msdn.microsoft.com/Forums/en-US/async/thread/163ef755-ff7b-4ea5-b226-bbe8ef5f4796)
Can someone help me?
Edit #1:
//When i click on this button, it loads the file and all canvas are drawn on canvasSegment.
private async void btnLoadClassExercises_Click(object sender, RoutedEventArgs e)
{
try
{
if (classModel != null)
{
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.FileTypeFilter.Add(".xml");
var file = await openPicker.PickSingleFileAsync();
classModel = xml.getControlClass(file);
auxCV = 0;
if (editClass)
CanvasSegment.Children.Clear();
foreach (Templates temp in classModel)
createCanvasSegment(temp);
editClass = true;
}
}
catch { }
}
// When i want to del some canvas, i put the removeCV = true and click on a canvas
private void btnDelExercise_Click(object sender, RoutedEventArgs e)
{
classEdit = classModel; //To debuf if the classModel.Count = 0
removeCV = true;
}
//When i click on a canvas, if removeCV = false, i draw the canvas on the main canvas(canvasClass), otherwise, i delete it. The problem is here. My modelClass.count here is = 0.
private void cv_PointerPressed(Templates temp, Canvas cv)
{
if (!removeCV)
{
foreach (StackPanel sp in CanvasSegment.Children)
foreach (Canvas c in sp.Children)
c.Opacity = 1;
cv.Opacity = 0.8;
canvasClass.Children.Clear();
for (int i = 0; i < temp.Shapes.Count; i++)
{
//Do some stuff with points and draws on CanvasClass later
rh.renderModel(new InkManager(), canvasClass, temp.Shapes[i], true, false);
}
}
else
{
for (int i = 0; i <= classModel.Count; i++)
if (classModel[i] == temp)
{
classModel.RemoveAt(i);
break;
}
CanvasSegment.Children.Clear();
auxCV = 0;
foreach (Templates tempModel in classModel)
createCanvasSegment(tempModel);
removeCV = false;
}
}
//I create a mini canvas foreach template in file.
private void createCanvasSegment(Templates temp)
{
Thickness thk = new Thickness();
thk.Right = 5;
thk.Bottom = 5;
if (temp != null || temp.Shapes != null)
{
if (auxCV == 0)
{
spcv = new StackPanel();
spcv.Orientation = Orientation.Horizontal;
spcv.Width = 540;
}
Canvas cv = new Canvas()
{
Background = new SolidColorBrush(Colors.Wheat),
Name = temp.Template,
Height = 73.125,
Width = 130,
Margin = thk,
};
cv.PointerPressed += (s, e) => cv_PointerPressed(temp, cv);
foreach (ShapePoints tempshapePoints in temp.Shapes)
{
ShapePoints tempS = tempShapePoints;
if (!removeCV)
{
//Do some stuff with points
}
rh.renderModel(new InkManager(), cv, tempS, true, false);
}
auxCV++;
spcv.Children.Add(cv);
if (auxCV == 1)
CanvasSegment.Children.Add(spcv);
else if (auxCV == 4)
auxCV = 0;
}
}
Since this is already in a UI thread's context, there's no need for AsyncContext.Run - just use:
private async void btnLoadClassExercises_Click(object sender, RoutedEventArgs e)
{
await loadFile());
// do some stuff with classModel
}
// Make this task returning (or even return the list instead of setting a local)
private async Task loadFile()
{
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.FileTypeFilter.Add(".xml");
var file = await openPicker.PickSingleFileAsync();
classModel = xml.getControlClass(file);
}

Categories