I followed Microsoft's instructions to implement MediaFrameReader API in order to process video captured by camera. The problem is I don't know why the bitmap doesn't show up in XAML, I can't save it to jpeg either (bitmap is not empty). I did convert the bitmap to proper format for XAML:
Here's my code to show the frame in XAML control (no error):
public void ShowSoftwareBitmap(SoftwareBitmap softwareBitmap, SoftwareBitmap backBuffer, Image imageElement, bool taskRunning)
{
try
{
//check if softwareBitmap is in proper format (Bgra8 and premultiplied alpha) to display to XAML Image control
if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8 ||
softwareBitmap.BitmapAlphaMode != BitmapAlphaMode.Premultiplied)
{
softwareBitmap = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
if(softwareBitmap != null)
{
softwareBitmap = Interlocked.Exchange(ref backBuffer, softwareBitmap);
softwareBitmap?.Dispose();
var task = imageElement.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
async () =>
{
if (taskRunning)
{
return;
}
taskRunning = true;
SoftwareBitmap latestBitmap;
while ((latestBitmap = Interlocked.Exchange(ref backBuffer, null)) != null)
{
SaveSoftwareBitmapToJpg(latestBitmap, 2);
var imageSource = (SoftwareBitmapSource)imageElement.Source;
await imageSource.SetBitmapAsync(latestBitmap);
latestBitmap.Dispose();
}
taskRunning = false;
}
);
}
}
catch (NullReferenceException e)
{
Debug.WriteLine("Backbuffer is empty." + e.Message);
}
}
Here's my code to save it to jpeg (error unsupported bitmap format):
public async void SaveSoftwareBitmapToJpg(SoftwareBitmap softwareBitmap, int _frameIndex)
{
StorageFolder captureFolder = await FileAccess();
StorageFile outputFile = await captureFolder.CreateFileAsync($"capture{_frameIndex}.jpg", CreationCollisionOption.FailIfExists);
using (IRandomAccessStream stream = await outputFile.OpenAsync(FileAccessMode.ReadWrite))
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
encoder.BitmapTransform.ScaledHeight = 300;
encoder.BitmapTransform.ScaledWidth = 300;
encoder.IsThumbnailGenerated = true;
try
{
await encoder.FlushAsync();
}
catch (Exception e)
{
switch (e.HResult)
{
case unchecked((int)0x88982F81): //WINCODEC_ERR_UNSUPPORTEDOPERATION
// If the encoder does not support writing a thumbnail, then try again
// but disable thumbnail generation.
encoder.IsThumbnailGenerated = false;
break;
default:
throw e;
}
}
//catch error generating thumbnail
if (encoder.IsThumbnailGenerated == false)
{
await encoder.FlushAsync();
}
}
}
I tested with your code snippet, if you invoking the above methods correctly with a correct SoftwareBitmap object, the above methods could work well. Simply for example:
<Image x:Name="imgshow" ></Image>
imgshow.Source = new SoftwareBitmapSource();
SoftwareBitmap _backBuffer = new SoftwareBitmap(BitmapPixelFormat.Bgra8, 300, 400);
ShowSoftwareBitmap(softwareBitmap, _backBuffer, imgshow, false);
So that issue may be not happened with the methods above. This could help you narrow the issue, you could check if something wrong when getting the SoftwareBitmap object. I saw you may copy code from ProcessFrame method which is in FrameRenderer class of the official sample. The official sample get the SoftwareBitmap by converting VideoMediaFrame, check if anything wrong when you converting.
Also please debug your code to see if taskRunning is false that your code do run into setting image source steps.
So here's how I changed my code:
In ShowSoftwareBitmap(), I used the same boolean taskRunning in other methods so it was changed crazily in run time. Just create another boolean to solve the problem.
In save SaveSoftwareBitmapToJpg(), the bitmap wasn't properly converted, it has to be Rgba8 or Rgba16 to print to JPEG, so the same conversion used for ShowSoftwareBitmap() wouldn't work. Use this instead:
try
{
if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Rgba16)
{
result = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Rgba16);
}
else
{
result = SoftwareBitmap.Copy(softwareBitmap);
}
}
Related
I am trying to use the Windows.Graphics.Capture namespace to capture screenshot from a comaptible application.
I based my code on the official examples here: MS sample screenshot code
The code in question is the following:
private async void btnCapture_Click(object sender, EventArgs e)
{
process = Process.GetProcesses().Where(p => p.ProcessName == "xxxxxxxxxxx").Single();
hwnd = process.MainWindowHandle;
GraphicsCaptureItem item = CaptureHelper.CreateItemForWindow(hwnd);
device = Direct3D11Helper.CreateDevice();
capture = new BasicCapture(device, item);
framePool = Direct3D11CaptureFramePool.Create(device, DirectXPixelFormat.B8G8R8A8UIntNormalized, 1, item.Size);
session = framePool.CreateCaptureSession(item);
session.StartCapture();
using (frame = framePool.TryGetNextFrame())
{
var bmp = await SoftwareBitmap.CreateCopyFromSurfaceAsync(frame.Surface);
StorageFolder pictureFolder = KnownFolders.CameraRoll;
StorageFile file = await pictureFolder.CreateFileAsync("test.png", CreationCollisionOption.ReplaceExisting);
using (var fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, fileStream);
encoder.SetSoftwareBitmap(bmp);
await encoder.FlushAsync();
}
}
}
If i put a breakpoint in line using (frame = framePool.TryGetNextFrame()) the frame is populated and the file is saved normally by the code that follows.
If i just run the app then frame is null and an error occurs. I cant use await on the framePool.TryGetNextFrame()
I feel that this is a Frankenstein code as i am learning the new capturing paradigm. Any help how to make this code work?
Eventually i will need this code to work with a timer taking screenshots at a regular interval.
Direct3D11CaptureFramePool has a FrameArrived event, which you could listen for before trying to get the frame:
var cts = new TaskCompletionSource<object>();
framePool.FrameArrived += (s, e) => cts.SetResult(null);
session.StartCapture();
await cts.Task;
// Frame should now be available
using (frame = framePool.TryGetNextFrame())
In the Windows OCR avalaible here : https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/OCR
In OcrCapturedImage.xaml.cs, I can't figure out why in the line :
using (var currentFrame = await mediaCapture.GetPreviewFrameAsync(videoFrame))
I'm getting the following issue :
System exception
System.ArgumentException: 'Parametre incorrect.
Invalid subtype'
Here's the full method :
private async void ExtractButton_Tapped(object sender, Windows.UI.Xaml.Input.TappedRoutedEventArgs e)
{
//Get information about the preview.
var previewProperties = mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
int videoFrameWidth = (int)previewProperties.Width;
int videoFrameHeight = (int)previewProperties.Height;
// Create the video frame to request a SoftwareBitmap preview frame.
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, videoFrameWidth, videoFrameHeight);
// Capture the preview frame.
using (var currentFrame = await mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame.
SoftwareBitmap bitmap = currentFrame.SoftwareBitmap;
OcrEngine ocrEngine = OcrEngine.TryCreateFromLanguage(ocrLanguage);
if (ocrEngine == null)
{
rootPage.NotifyUser(ocrLanguage.DisplayName + " is not supported.", NotifyType.ErrorMessage);
return;
}
var imgSource = new WriteableBitmap(bitmap.PixelWidth, bitmap.PixelHeight);
bitmap.CopyToBuffer(imgSource.PixelBuffer);
PreviewImage.Source = imgSource;
var ocrResult = await ocrEngine.RecognizeAsync(bitmap);
}
}
Thanks for any help
I want to add a custom tile to the Microsoft Band through Microsoft Band SDK in a UWP app for Windows Phone. Here is my sample code.
private async void ButtonBase_OnClick(object sender, RoutedEventArgs e)
{
try
{
// Get the list of Microsoft Bands paired to the phone.
var pairedBands = await BandClientManager.Instance.GetBandsAsync();
if (pairedBands.Length < 1)
{
Debug.WriteLine("This sample app requires a Microsoft Band paired to your device.Also make sure that you have the latest firmware installed on your Band, as provided by the latest Microsoft Health app.");
return;
}
// Connect to Microsoft Band.
using (var bandClient = await BandClientManager.Instance.ConnectAsync(pairedBands[0]))
{
// Create a Tile with a TextButton on it.
var myTileId = new Guid("12408A60-13EB-46C2-9D24-F14BF6A033C6");
var myTile = new BandTile(myTileId)
{
Name = "My Tile",
TileIcon = await LoadIcon("ms-appx:///Assets/SampleTileIconLarge.png"),
SmallIcon = await LoadIcon("ms-appx:///Assets/SampleTileIconSmall.png")
};
// Remove the Tile from the Band, if present. An application won't need to do this everytime it runs.
// But in case you modify this sample code and run it again, let's make sure to start fresh.
await bandClient.TileManager.RemoveTileAsync(myTileId);
// Create the Tile on the Band.
await bandClient.TileManager.AddTileAsync(myTile);
// Subscribe to Tile events.
}
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
}
private async Task<BandIcon> LoadIcon(string uri)
{
StorageFile imageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri(uri));
using (IRandomAccessStream fileStream = await imageFile.OpenAsync(FileAccessMode.Read))
{
WriteableBitmap bitmap = new WriteableBitmap(1, 1);
await bitmap.SetSourceAsync(fileStream);
return bitmap.ToBandIcon();
}
}
If I run this code nothing happend. The app connected to Microsoft Band, but is not able to add a tile. The method AddTileAsync(myTile); Returns false and doesn't add a tile to the Microsoft Band.
If I try this code in a Windows Phone 8.1 app it works, but not in the UWP app.
Any ideas?
Update
Here is the sample app as download. Maybe this can help.
maybe this would help, coming from the documentation of MS Band
using Microsoft.Band.Tiles;
...
try
{
IEnumerable<BandTile> tiles = await bandClient.TileManager.GetTilesAsync();
}
catch (BandException ex)
{
//handle exception
}
//determine if there is space for tile
try
{
int tileCapacity = await bandClient.TileManager.GetRemainingTileCapacityAsync();
}
catch (BandException ex)
{
//handle ex
}
//create tile
WriteAbleBitmap smallIconBit = new WriteAbleBitmap(24, 24);
BandIcon smallIcon = smallIconBit.ToBandIcon();
WriteAbleBitmap largeIconBit = new WriteAbleBitmap(48, 48);//46, 46 for MS band 1
BandIcon largeIcon = largeIconBit.ToBandIcon();
Guid guid = Guid.NewGuid();
BandTile tile = new BandTile(guid)
{
//enable Badging
IsBadgingEnabled = true,
Name = "MYNAME"
SmallIcon = smallIcon;
TileIcon = largeIcon;
};
try
{
if(await bandClient.TileManager.AddTileAsync(tile))
{
///console print something
}
}
catch(BandException ex)
{
//blabla handle
}
I think the issue may be you're setting the writeable bitmap size to (1,1)?
I have this method working:
public static class BandIconUtil
{
public static async Task<BandIcon> FromAssetAsync(string iconFileName, int size = 24)
{
string uri = "ms-appx:///" + iconFileName;
StorageFile imageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri(uri, UriKind.RelativeOrAbsolute));
using (IRandomAccessStream fileStream = await imageFile.OpenAsync(FileAccessMode.Read))
{
WriteableBitmap bitmap = new WriteableBitmap(size, size);
await bitmap.SetSourceAsync(fileStream);
return bitmap.ToBandIcon();
}
}
}
Im currently writing my first app for WP8.1. The app just need to scan a barcode.
public async void ScanBarcodeAsync(Windows.Storage.StorageFile Afile)
{
WriteableBitmap bitmap;
BitmapDecoder decoder;
using (IRandomAccessStream str = await Afile.OpenReadAsync())
{
decoder = await BitmapDecoder.CreateAsync(str);
bitmap = new WriteableBitmap(Convert.ToInt32(decoder.PixelWidth),
Convert.ToInt32(decoder.PixelHeight));
await bitmap.SetSourceAsync(str);
}
ZXing.BarcodeReader reader = new BarcodeReader();
/*reader.Options.PossibleFormats = new ZXing.BarcodeFormat[]
{
ZXing.BarcodeFormat.CODE_128,
ZXing.BarcodeFormat.CODE_39
};*/
reader.Options.TryHarder = true;
reader.AutoRotate = true;
var results = reader.Decode(bitmap);
if (results != null)
{
edtBarcode.Text = results.Text;
}
else
{
edtBarcode.Text = "Error";
}
}
The file for this method is created this way
async void StartCapture()
{
var cameraID = await GetCameraID(Windows.Devices.Enumeration.Panel.Back);
MediaCaptureInitializationSettings settings = new MediaCaptureInitializationSettings();
settings.PhotoCaptureSource = Windows.Media.Capture.PhotoCaptureSource.VideoPreview;
settings.StreamingCaptureMode = Windows.Media.Capture.StreamingCaptureMode.Video;
settings.AudioDeviceId = string.Empty;
settings.VideoDeviceId = cameraID.Id;
captureManager = new MediaCapture(); //Define MediaCapture object
await captureManager.InitializeAsync(settings); //Initialize MediaCapture and
var focusSettings = new FocusSettings();
focusSettings.AutoFocusRange = AutoFocusRange.Macro;
focusSettings.Mode = FocusMode.Auto;
focusSettings.WaitForFocus = true;
focusSettings.DisableDriverFallback = false;
captureManager.VideoDeviceController.FocusControl.Configure(focusSettings);
captureManager.SetPreviewRotation(VideoRotation.Clockwise90Degrees);
capturePreview.Source = captureManager; //Start preiving on CaptureElement
await captureManager.StartPreviewAsync(); //Start camera capturing
}
async private void Capture_Photo_Click(object sender, RoutedEventArgs e)
{
//Create JPEG image Encoding format for storing image in JPEG type
ImageEncodingProperties imgFormat = ImageEncodingProperties.CreateJpeg();
// create storage file in local app storage
if (ImageIndex > 0)
{
StorageFile delFile = await ApplicationData.Current.LocalFolder.GetFileAsync("Photo" + Convert.ToString(ImageIndex - 1) + ".jpg");
await delFile.DeleteAsync();
}
file = await ApplicationData.Current.LocalFolder.CreateFileAsync("Photo" + Convert.ToString(ImageIndex++) + ".jpg", CreationCollisionOption.ReplaceExisting);
// take photo and store it on file location.
await captureManager.CapturePhotoToStorageFileAsync(imgFormat, file);
//// create storage file in Picture Library
//StorageFile file = await KnownFolders.PicturesLibrary.CreateFileAsync("Photo.jpg",CreationCollisionOption.GenerateUniqueName);
// Get photo as a BitmapImage using storage file path.
bmpImage = new BitmapImage();
bmpImage.CreateOptions = BitmapCreateOptions.None;
bmpImage.UriSource = new Uri(file.Path);
ImageTaken = true;
Frame.Navigate(typeof(MainPage));
Frame.Navigate(typeof(MainPage));
}
I dont get an error running this code. But the result from ZXing is always null. The ScanBarcodeAsync-Function is directly run after taking the picture. The image the app takes is not sharp. Can this cause the problem?
Im happy to get any suggestions to solve my problem.
I have developed an application the continuous read image stream from a DSLR camera.
while (!liveViewExit)
{
// Create a Memory Stream
stream = new IntPtr();
// Get the bitmap image from the DSLR
bmp = GetEvfImage(stream);
if (bmp != null)
{
// Crop the image.
picImage = (System.Drawing.Image)bmp.Clone(new Rectangle(10, 0, 492, 768), bmp.PixelFormat);
try
{
if (picImage != null)
this.picLiveView.Image = (System.Drawing.Image)picImage.Clone();
}
catch (Exception ex)
{
Utility.HandleError(ex);
}
}
}
After running a while, I have this error for this line of code:
this.picLiveView.Image = (System.Drawing.Image)picImage.Clone();
Object is currently in use elsewhere.( at System.Drawing.Image.get_FrameDimensionsList()
at System.Drawing.ImageAnimator.CanAnimate(Image image)
at System.Drawing.ImageAnimator.ImageInfo..ctor(Image image)
at System.Drawing.ImageAnimator.Animate(Image image, EventHandler onFrameChangedHandler)
at System.Windows.Forms.PictureBox.Animate(Boolean animate)
at System.Windows.Forms.PictureBox.Animate()
at System.Windows.Forms.PictureBox.InstallNewImage(Image value, ImageInstallationType installationType)
at System.Windows.Forms.PictureBox.set_Image(Image value)
I think the picLiveView PictureBox control is not yet ready to accept new image.
Any idea how on detect if PictureBox is still in used.
// Added:
It is a single thread. I think the picturebox is not fast enough to process the picture object in the while loop.
Are multiple threads updating the picLiveView image? If so that would explain this problem. Just use one thread instead, and serialize the update - alternatively you could use a lock to access picLiveView:
private readonly object myLock = new object();
...
if (picImage != null)
{
lock(myLock)
{
this.picLiveView.Image = (System.Drawing.Image)picImage.Clone();
}
}
I know am late...but try this, if someone have same issue..
if (bmp != null)
{
// Crop the image.
picImage = (System.Drawing.Image)bmp.Clone(new Rectangle(10, 0, 492, 768), bmp.PixelFormat);
**Bitmap img = new Bitmap(picImage);
picImage.Dispose();
picImage = null;**
try
{
if (picImage != null)
**this.picLiveView.Image = img;**
}
catch (Exception ex)
{
Utility.HandleError(ex);
}
}