I have a function that works great when opening JPG and PNG files, but locks my computer up when opening a CR2 file. The CRS files are coming from my Canon EOS Rebel T6 Camera and they are big, see:
but I feel like it should not lock my computer up and it should not move this slow.
here is my code :
public async Task OpenFile()
{
OriginalImage = null;
await SetProcessBarInformation(0, "Openning Image file");
OpenFileDialog _openFileDialog = new OpenFileDialog();
if (_openFileDialog.ShowDialog() == DialogResult.OK)
{
try
{
await SetProcessBarInformation(20, "Openning Image file");
FileName = System.IO.Path.GetFileName(_openFileDialog.FileName);
BitmapImage tempOriginalImage = await GetPicture(_openFileDialog.FileName);
Height = (int)tempOriginalImage.Height;
Width = (int)tempOriginalImage.Width;
await SetProcessBarInformation(80, "Openning Image file");
await Task.Run(() =>
{
OriginalImage = tempOriginalImage;
});
}
catch (Exception ex)
{
ErrorMessage = ex.Message;
}
}
protected async Task<BitmapImage> GetPicture(string tempImage)
{
BitmapImage timage = null;
await Task.Run(() =>
{
timage = new BitmapImage(new Uri(tempImage));
timage.Freeze();
});
return timage;
}
I'm using media composition to preview videos and to generate streams. I pass a MediaEncoding Profile in which the video container I am assigning the bitrate has to apply when previewing. But it's not working. Here's the code:
MediaEncodingProfile profile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.HD720p);
profile.Video.Bitrate = 1000000; // Any Bitrate
var stream = VideoComposition.GenerateMediaStreamSource(profile);
videoPlayer.SetMediaStreamSource(stream);
By testing, there is no effect in the MediaStreamSource when we assign MediaEncodingProfile.Video.Bitrate in your scenario.
You could use Windows.Media.Transcoding APIs to transcode video files from one format to another to show the effect of assigning bitrate.
Please check the following code as a sample:
private MediaComposition VideoComposition;
private MediaStreamSource mediaStreamSource;
private StorageFile destination;
……
private async void button_Click(object sender, RoutedEventArgs e)
{
var picker = new Windows.Storage.Pickers.FileOpenPicker();
picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
picker.FileTypeFilter.Add(".mp4");
Windows.Storage.StorageFile pickedFile = await picker.PickSingleFileAsync();
if (pickedFile == null)
{
return;
}
StorageFolder localFolder = ApplicationData.Current.LocalFolder;
destination = await localFolder.CreateFileAsync("destination.mp4",CreationCollisionOption.ReplaceExisting);
MediaEncodingProfile profile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.HD720p);
MediaTranscoder transcoder = new MediaTranscoder();
profile.Video.Bitrate = 50000; // Any Bitrate
PrepareTranscodeResult prepareOp = await
transcoder.PrepareFileTranscodeAsync(pickedFile, destination, profile);
if (prepareOp.CanTranscode)
{
var transcodeOp = prepareOp.TranscodeAsync();
transcodeOp.Completed +=
new AsyncActionWithProgressCompletedHandler<double>(TranscodeComplete);
}
else
{
switch (prepareOp.FailureReason)
{
case TranscodeFailureReason.CodecNotFound:
System.Diagnostics.Debug.WriteLine("Codec not found.");
break;
case TranscodeFailureReason.InvalidProfile:
System.Diagnostics.Debug.WriteLine("Invalid profile.");
break;
default:
System.Diagnostics.Debug.WriteLine("Unknown failure.");
break;
}
}
}
private async void TranscodeComplete(IAsyncActionWithProgress<double> asyncInfo, AsyncStatus asyncStatus)
{
await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () => {
var stream = await destination.OpenAsync(Windows.Storage.FileAccessMode.Read);
videoPlayer.SetSource(stream, destination.ContentType);
});
}
You could delete the destination file if you do not want to save the file.
I want to open multiple audio files in UWP using the FileOpenPicker but I am getting an error that I cant Cannot Convert. How can I fix this?
And if this is fixed, will all the audio files play in order or all at the same time?
public MainPage()
{
this.InitializeComponent();
}
MediaSource media_source;
MediaPlayer media_player;
public async System.Threading.Tasks.Task OpenfileAsync()
{
var filePicker = new Windows.Storage.Pickers.FileOpenPicker();
filePicker.FileTypeFilter.Add(".mp3");
filePicker.FileTypeFilter.Add(".mp4");
filePicker.FileTypeFilter.Add(".ogg");
filePicker.FileTypeFilter.Add(".wav");
filePicker.FileTypeFilter.Add(".wma");
filePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.MusicLibrary;
StorageFile file = await filePicker.PickSingleFileAsync();
if (file != null)
{
media_source = MediaSource.CreateFromStorageFile(file);
media_player = new MediaPlayer();
media_player.Source = media_source;
mediaPlayerElement.SetMediaPlayer(media_player);
media_player.Play();
}
}
private async void Select_track_Click(object sender, RoutedEventArgs e)
{
await OpenfileAsync();
}
public async System.Threading.Tasks.Task OpenMultipleAsync()
{
var filePicker = new Windows.Storage.Pickers.FileOpenPicker();
filePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.MusicLibrary;
filePicker.FileTypeFilter.Add(".mp3");
filePicker.FileTypeFilter.Add(".mp4");
filePicker.FileTypeFilter.Add(".ogg");
filePicker.FileTypeFilter.Add(".wav");
filePicker.FileTypeFilter.Add(".wma");
filePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.MusicLibrary;
StorageFile file = await filePicker.PickMultipleFilesAsync();
if (file != null)
{
media_source = MediaSource.CreateFromStorageFile(file);
media_player = new MediaPlayer();
media_player.Source = media_source;
mediaPlayerElement.SetMediaPlayer(media_player);
media_player.Play();
}
}
private async void playlist_Click(object sender, RoutedEventArgs e)
{
await OpenMultipleAsync();
}
I am getting the error at StorageFile file = await filepicker.PickmultiplefilesAsync();
The FileOpenPicker.PickMultipleFilesAsync method has the following signature:
IAsyncOperation<IReadOnlyList<StorageFile>> PickMultipleFilesAsync()
In contrast to PickFileAsync it returns a IReadOnlyList<StorageFile>, so you will actually get a list of multiple files the user selected. You should update the code like this:
var files = await filePicker.PickMultipleFilesAsync();
foreach (var file in files)
{
if (file != null)
{
media_source = MediaSource.CreateFromStorageFile(file);
media_player = new MediaPlayer();
media_player.Source = media_source;
mediaPlayerElement.SetMediaPlayer(media_player);
media_player.Play();
}
}
This solution will play all the sounds at once. For one by one playback you can use #touseefbsb solution :-) .
for playing a list of files its best that you use MediaPlaybackList
Also you only need to set the SuggestedStartLocation once, and when you use PickMultipleFilesAsync() you get a List of files returned so you need that iterate through that list to get all files and add them to your MediaPlaybackList
Modify your OpenMultipleAsync method like this :
public async System.Threading.Tasks.Task OpenMultipleAsync()
{
var filePicker = new Windows.Storage.Pickers.FileOpenPicker();
filePicker.FileTypeFilter.Add(".mp3");
filePicker.FileTypeFilter.Add(".mp4");
filePicker.FileTypeFilter.Add(".ogg");
filePicker.FileTypeFilter.Add(".wav");
filePicker.FileTypeFilter.Add(".wma");
filePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.MusicLibrary;
_mediaPlaybackList = new MediaPlaybackList();
var files = await filePicker.PickMultipleFilesAsync();
foreach (var file in files)
{
var mediaPlaybackItem = new MediaPlaybackItem(MediaSource.CreateFromStorageFile(file));
_mediaPlaybackList.Items.Add(mediaPlaybackItem);
}
_mediaPlayer = new MediaPlayer();
_mediaPlayer.Source = _mediaPlaybackList;
mediaPlayerElement.SetMediaPlayer(_mediaPlayer);
}
More details about MediaPlaybackItem can be seen here
and to answer 'will these media files play together at the same time or one after the other' : they will play one after the other in a row, that is the purpose if MediaPlaybackList, it supports gapless playback for playlists.
In the Windows OCR avalaible here : https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/OCR
In OcrCapturedImage.xaml.cs, I can't figure out why in the line :
using (var currentFrame = await mediaCapture.GetPreviewFrameAsync(videoFrame))
I'm getting the following issue :
System exception
System.ArgumentException: 'Parametre incorrect.
Invalid subtype'
Here's the full method :
private async void ExtractButton_Tapped(object sender, Windows.UI.Xaml.Input.TappedRoutedEventArgs e)
{
//Get information about the preview.
var previewProperties = mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
int videoFrameWidth = (int)previewProperties.Width;
int videoFrameHeight = (int)previewProperties.Height;
// Create the video frame to request a SoftwareBitmap preview frame.
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, videoFrameWidth, videoFrameHeight);
// Capture the preview frame.
using (var currentFrame = await mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame.
SoftwareBitmap bitmap = currentFrame.SoftwareBitmap;
OcrEngine ocrEngine = OcrEngine.TryCreateFromLanguage(ocrLanguage);
if (ocrEngine == null)
{
rootPage.NotifyUser(ocrLanguage.DisplayName + " is not supported.", NotifyType.ErrorMessage);
return;
}
var imgSource = new WriteableBitmap(bitmap.PixelWidth, bitmap.PixelHeight);
bitmap.CopyToBuffer(imgSource.PixelBuffer);
PreviewImage.Source = imgSource;
var ocrResult = await ocrEngine.RecognizeAsync(bitmap);
}
}
Thanks for any help
I'm trying to make an app that use the camera to record a video and process the images of the video. Here is what I want. First, my app records a 10 second video with Torch. Second, I use a method to playback the video to see what I record.
I'm stuck on three things.
How can I convert my video into individual frames (images)?
Is it possible to asynchronously convert the video while it is being recorded?
When I do convert the video into individual frames, how do I work with them? Are they JPEGs? Can I simply display them as images? Etc.
Main code:
using System;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Navigation;
namespace App3
{
public sealed partial class MainPage : Page
{
DispatcherTimer D;
double basetimer = 0;
public MainPage()
{
this.InitializeComponent();
this.NavigationCacheMode = NavigationCacheMode.Required;
D = new DispatcherTimer();
D.Interval = new TimeSpan(0, 0, 1);
D.Tick += timer_Tick;
txt.Text = basetimer.ToString();
Play.IsEnabled = false;
}
public Library Library = new Library();
public object PreviewImage { get; private set; }
void timer_Tick(object sender, object e)
{
basetimer = basetimer - 1;
txt.Text = basetimer.ToString();
if (basetimer == 0)
{
D.Stop();
Preview.Source = null;
Library.Stop();
Record.IsEnabled = false;
Play.IsEnabled = true;
Clear.IsEnabled = true;
if (Library._tc.Enabled)
{
Library._tc.Enabled = false;
}
}
}
private void Record_Click(object sender, RoutedEventArgs e)
{
if (Library.Recording)
{
Preview.Source = null;
Library.Stop();
Record.Icon = new SymbolIcon(Symbol.Video);
}
else
{
basetimer = 11;
D.Start();
//D.Tick += timer_Tick;
Display.Source = null;
Library.Record(Preview);
Record.Icon = new SymbolIcon(Symbol.VideoChat);
Record.IsEnabled = false;
Play.IsEnabled = false;
}
}
private async void Play_Click(object sender, RoutedEventArgs e)
{
await Library.Play(Dispatcher, Display);
//Extract_Image_From_Video(Library.buffer);
}
private void Clear_Click(object sender, RoutedEventArgs e)
{
Display.Source = null;
Record.Icon = new SymbolIcon(Symbol.Video);
txt.Text = "0";
basetimer= 0;
Play.IsEnabled = false;
Record.IsEnabled =true;
if (Library.capture != null)
{
D.Stop();
Library.Recording = false;
Preview.Source = null;
Library.capture.Dispose();
Library.capture = null;
basetimer = 11;
}
}
}
}
Library Class:
using System;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
using Windows.Devices.Enumeration;
using Windows.Media.Capture;
using Windows.Media.Devices;
using Windows.Media.MediaProperties;
using Windows.Storage;
using Windows.Storage.Streams;
using Windows.UI.Core;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Media.Imaging;
using Windows.Graphics.Imaging;
using Emgu.CV.Structure;
using Emgu.CV;
using System.Collections.Generic;
public class Library
{
private const string videoFilename = "video.mp4";
private string filename;
public MediaCapture capture;
public InMemoryRandomAccessStream buffer;
public static bool Recording;
public TorchControl _tc;
public int basetimer ;
public async Task<bool> init()
{
if (buffer != null)
{
buffer.Dispose();
}
buffer = new InMemoryRandomAccessStream();
if (capture != null)
{
capture.Dispose();
}
try
{
if (capture == null)
{
var allVideoDevices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);
DeviceInformation cameraDevice =
allVideoDevices.FirstOrDefault(x => x.EnclosureLocation != null &&
x.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Back);
capture = new MediaCapture();
var mediaInitSettings = new MediaCaptureInitializationSettings { VideoDeviceId = cameraDevice.Id };
// Initialize
try
{
await capture.InitializeAsync(mediaInitSettings);
var videoDev = capture.VideoDeviceController;
_tc = videoDev.TorchControl;
Recording = false;
_tc.Enabled = false;
}
catch (UnauthorizedAccessException)
{
Debug.WriteLine("UnauthorizedAccessExeption>>");
}
catch (Exception ex)
{
Debug.WriteLine("Exception when initializing MediaCapture with {0}: {1}", cameraDevice.Id, ex.ToString());
}
}
capture.Failed += (MediaCapture sender, MediaCaptureFailedEventArgs errorEventArgs) =>
{
Recording = false;
_tc.Enabled = false;
throw new Exception(string.Format("Code: {0}. {1}", errorEventArgs.Code, errorEventArgs.Message));
};
}
catch (Exception ex)
{
if (ex.InnerException != null && ex.InnerException.GetType() == typeof(UnauthorizedAccessException))
{
throw ex.InnerException;
}
throw;
}
return true;
}
public async void Record(CaptureElement preview)
{
await init();
preview.Source = capture;
await capture.StartPreviewAsync();
await capture.StartRecordToStreamAsync(MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto), buffer);
if (Recording) throw new InvalidOperationException("cannot excute two records at the same time");
Recording = true;
_tc.Enabled = true;
}
public async void Stop()
{
await capture.StopRecordAsync();
Recording = false;
_tc.Enabled = false;
}
public async Task Play(CoreDispatcher dispatcher, MediaElement playback)
{
IRandomAccessStream video = buffer.CloneStream();
if (video == null) throw new ArgumentNullException("buffer");
StorageFolder storageFolder = Windows.ApplicationModel.Package.Current.InstalledLocation;
if (!string.IsNullOrEmpty(filename))
{
StorageFile original = await storageFolder.GetFileAsync(filename);
await original.DeleteAsync();
}
await dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () =>
{
StorageFile storageFile = await storageFolder.CreateFileAsync(videoFilename, CreationCollisionOption.GenerateUniqueName);
filename = storageFile.Name;
using (IRandomAccessStream fileStream = await storageFile.OpenAsync(FileAccessMode.ReadWrite))
{
await RandomAccessStream.CopyAndCloseAsync(video.GetInputStreamAt(0), fileStream.GetOutputStreamAt(0));
await video.FlushAsync();
video.Dispose();
}
IRandomAccessStream stream = await storageFile.OpenAsync(FileAccessMode.Read);
playback.SetSource(stream, storageFile.FileType);
playback.Play();
});
}
I ended up using MediaToolkit to solve a similar problem after having a ton of trouble with Accord.
I needed to save an image for every second of a video:
using (var engine = new Engine())
{
var mp4 = new MediaFile { Filename = mp4FilePath };
engine.GetMetadata(mp4);
var i = 0;
while (i < mp4.Metadata.Duration.TotalSeconds)
{
var options = new ConversionOptions { Seek = TimeSpan.FromSeconds(i) };
var outputFile = new MediaFile { Filename = string.Format("{0}\\image-{1}.jpeg", outputPath, i) };
engine.GetThumbnail(mp4, outputFile, options);
i++;
}
}
Hope this helps someone some day.
UPDATE for .NET 5:
Recently, I have needed to update this code to work in .NET 5. To do so, I am using MediaToolkit.NetCore, which has been in preview for over a year. Also note: you will need to make the latest ffmpeg, including all 3 .exe files (ffmpeg, ffplay, ffprobe) available to your app.
Without further ado, here is the updated code:
// _env is the injected IWebHostEnvironment
// _tempPath is temporary file storage
var ffmpegPath = Path.Combine(_env.ContentRootPath, "<path-to-ffmpeg.exe>");
var mediaToolkitService = MediaToolkitService.CreateInstance(ffmpegPath);
var metadataTask = new FfTaskGetMetadata(_tempFile);
var metadata = await mediaToolkitService.ExecuteAsync(metadataTask);
var i = 0;
while (i < metadata.Metadata.Streams.First().DurationTs)
{
var outputFile = string.Format("{0}\\image-{1:0000}.jpeg", _imageDir, i);
var thumbTask = new FfTaskSaveThumbnail(_tempFile, outputFile, TimeSpan.FromSeconds(i));
_ = await mediaToolkitService.ExecuteAsync(thumbTask);
i++;
}
I figured this out just yesterday.
Here is full and easy to understand example with picking video file and saving snapshot in 1st second of video.
You can take parts that fits your project and change some of them (i.e. getting video resolution from camera)
1) and 3)
TimeSpan timeOfFrame = new TimeSpan(0, 0, 1);
//pick mp4 file
var picker = new Windows.Storage.Pickers.FileOpenPicker();
picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
picker.FileTypeFilter.Add(".mp4");
StorageFile pickedFile = await picker.PickSingleFileAsync();
if (pickedFile == null)
{
return;
}
///
//Get video resolution
List<string> encodingPropertiesToRetrieve = new List<string>();
encodingPropertiesToRetrieve.Add("System.Video.FrameHeight");
encodingPropertiesToRetrieve.Add("System.Video.FrameWidth");
IDictionary<string, object> encodingProperties = await pickedFile.Properties.RetrievePropertiesAsync(encodingPropertiesToRetrieve);
uint frameHeight = (uint)encodingProperties["System.Video.FrameHeight"];
uint frameWidth = (uint)encodingProperties["System.Video.FrameWidth"];
///
//Use Windows.Media.Editing to get ImageStream
var clip = await MediaClip.CreateFromFileAsync(pickedFile);
var composition = new MediaComposition();
composition.Clips.Add(clip);
var imageStream = await composition.GetThumbnailAsync(timeOfFrame, (int)frameWidth, (int)frameHeight, VideoFramePrecision.NearestFrame);
///
//generate bitmap
var writableBitmap = new WriteableBitmap((int)frameWidth, (int)frameHeight);
writableBitmap.SetSource(imageStream);
//generate some random name for file in PicturesLibrary
var saveAsTarget = await KnownFolders.PicturesLibrary.CreateFileAsync("IMG" + Guid.NewGuid().ToString().Substring(0, 4) + ".jpg");
//get stream from bitmap
Stream stream = writableBitmap.PixelBuffer.AsStream();
byte[] pixels = new byte[(uint)stream.Length];
await stream.ReadAsync(pixels, 0, pixels.Length);
using (var writeStream = await saveAsTarget.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, writeStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)writableBitmap.PixelWidth,
(uint)writableBitmap.PixelHeight,
96,
96,
pixels);
await encoder.FlushAsync();
using (var outputStream = writeStream.GetOutputStreamAt(0))
{
await outputStream.FlushAsync();
}
}
If you want to display frames in xaml Image, you should use imageStream
BitmapImage bitmapImage = new BitmapImage();
bitmapImage.SetSource(imageStream);
XAMLImage.Source = bitmapImage;
If you want to extract more frames, there is also composition.GetThumbnailsAsync
2) Use your mediaCapture, when your timer is ticking
EDIT:
used includes:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices.WindowsRuntime;
using System.Threading;
using System.Threading.Tasks;
using Windows.Graphics.Imaging;
using Windows.Media.Editing;
using Windows.Storage;
using Windows.UI.Xaml.Media.Imaging;
Use ffmpeg and install Accord.Video.FFMPEG
using (var vFReader = new VideoFileReader())
{
vFReader.Open("video.mp4");
for (int i = 0; i < vFReader.FrameCount; i++)
{
Bitmap bmpBaseOriginal = vFReader.ReadVideoFrame();
}
vFReader.Close();
}
Another one way to get it :
I used FFMpegCore, official docker image with .net core 3.1 + ubuntu (list of available images)
Dockerfile :
FROM mcr.microsoft.com/dotnet/runtime:3.1-bionic
RUN apt-get update && apt-get install -y ffmpeg libgdiplus
COPY bin/Release/netcoreapp3.1/publish/ App/
WORKDIR /App
ENTRYPOINT ["dotnet", "YouConsoleAppNameHere.dll"]
short code version:
GlobalFFOptions.Configure(new FFOptions { BinaryFolder = "/usr/bin", TemporaryFilesFolder = "/tmp" }); //configuring ffmpeg location
string filePath = AppContext.BaseDirectory + "sample.mp4";
FFMpegArguments.FromFileInput(filePath).OutputToFile("tmp/Video/Frame%05d.png", true, Options => { Options.WithVideoCodec(VideoCodec.Png); }).ProcessSynchronously();
extended version (with some console logs):
using FFMpegCore;
using FFMpegCore.Enums;
...
GlobalFFOptions.Configure(new FFOptions { BinaryFolder = "/usr/bin", TemporaryFilesFolder = "/tmp" }); //configuring ffmpeg location
string filePath = AppContext.BaseDirectory + "sample.mp4";
Console.WriteLine(filePath) ;
Console.WriteLine(File.Exists(filePath));
var mediaInfo = FFProbe.Analyse(filePath);
Console.WriteLine("mp4 duration : " + mediaInfo.Duration);
Directory.CreateDirectory("tmp");
Directory.CreateDirectory("tmp/Video");
Console.WriteLine("started " + DateTime.Now.ToLongTimeString());
FFMpegArguments.FromFileInput(filePath).OutputToFile("tmp/Video/Frame%05d.png", true, Options => { Options.WithVideoCodec(VideoCodec.Png); }).ProcessSynchronously();
Console.WriteLine("processed " + DateTime.Now.ToLongTimeString());
Console.WriteLine(string.Join(", ", Directory.EnumerateFiles("tmp/Video/")));
As a result - png files will be extracted to tmp/Video folder. Of course, you can do the same without docker if you need.