We Keep Coding

How to create a IDirect3DSurface from an IBuffer or byte array in UWP

I want to create a video from a few RenderTargetBitmaps in UWP. I am doing that by using MediaClips.
From RenderTargetBitmap i can get an IBuffer or byte array of pixels.
To create a MediaClip I need either an image file or an IDirect3DSurface.
Creating an image just to create a clip is very expensive, so I thought of using IDirect3DSurface.
How can I do this?
I have tried this:
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(RenderedGrid, 100, 100);
IBuffer pixels = await renderTargetBitmap.GetPixelsAsync();
var values = Enum.GetValues(typeof(DirectXPixelFormat));
CanvasBitmap bitmap=null;
foreach (DirectXPixelFormat format in values)
{
try
{
videoClip = new MediaComposition();
bitmap = CanvasBitmap.CreateFromBytes(myWidget.Device, pixels, renderTargetBitmap.PixelWidth, renderTargetBitmap.PixelHeight, format);
StorageFile video2 = await storageFolder.CreateFileAsync("video2" + ".mp4", CreationCollisionOption.ReplaceExisting);
MediaClip d = MediaClip.CreateFromSurface(bitmap, DateTime.Now - previousFrame+new TimeSpan(100));
videoClip.Clips.Add(d);
await videoClip.RenderToFileAsync(video2);
break;
}
catch(Exception e)
{
}
}
I try all the formats in DirectXPixelFormat but none works.
I have a CanvasControl named myWidget that is empty.
I create a CanvasBitmap from Ibuffer (CanvasBitmap implements IDirect3DSurface)
Create a Mediaclip from CanvasBitmap
Add it to MediaComposition.
Then I try to render to video file.When i try to save to a file it throws an error
System.Runtime.InteropServices.COMException Stream is not in a state
to handle the request.
EDIT:
I figured out where the problem is, but not why and not how to fix it.
await videoClip.SaveAsync(video2);
videoClip= await MediaComposition.LoadAsync(video2);
var x=await videoClip.RenderToFileAsync(video2);
Now with these three lines i can save the video, but using only the third line it throws the error above. I cannot make sense of it. Why does saving and loading fix the problem??

Most probable reason is that, CanvasBitmap has an underlying IDirce3DSurface object as well as image data like byte[] or something else , though I am not sure about this.
If that's true, then creating a CanvasBitmap from byte[] or IBuffer won't effect the underlying IDirect3DSurface, the image part will be constructed only. You can see that by saving that image on the disk, it gives no error.
But I think there's a workaround if you want to skip saving data on the disk:
You can contruct the underlying IDirect3DSurface if you do Offscreen Drawing to a CanvasRenderTarget.
So, you can use the CanvasBitmap to construct a CanvasRenderTarget and then use that CanvasRenderTarget to contruct a MediaClip:
CanvasRenderTarget rendertarget;
using (CanvasBitmap canvas = CanvasBitmap.CreateFromBytes(CanvasDevice.GetSharedDevice(), pixels, renderTargetBitmap.PixelWidth, renderTargetBitmap.PixelHeight, format))
{
rendertarget = new CanvasRenderTarget(CanvasDevice.GetSharedDevice(), canvas.SizeInPixels.Width, canvas.SizeInPixels.Height, 96);
using (CanvasDrawingSession ds = rendertarget.CreateDrawingSession())
{
ds.Clear(Colors.Black);
ds.DrawImage(canvas);
}
}
MediaClip d = MediaClip.CreateFromSurface(renderTarget, TimeSpan.FromMilliseconds(80));
mc.Clips.Add(m);

The MediaComposition.RenderToFileAsync Method saves the composition to a video file that can be played back with standard media players. From the error info, it seems the stream content is not correct media data and can not be render into a video file directly.
So, to create a video from a few RenderTargetBitmaps in UWP, the way to use an image file should be your choice. using MediaClip.CreateFromImageFileAsync method by saving the RenderTargetBitmap into a file then using it to create a video.
private async void CreateVideoByConvertRenderBitmapToFile()
{
var folder = await ApplicationData.Current.LocalFolder.CreateFolderAsync("Test",
CreationCollisionOption.ReplaceExisting);
var composition = new MediaComposition();
for (int i = 0; i < 5; i++)
{
RenderTargetBitmap render = new RenderTargetBitmap();
await render.RenderAsync(RenderGrid);
MyImage.Source = render;
var pixel = await render.GetPixelsAsync();
var file = await folder.CreateFileAsync("test.png", CreationCollisionOption.GenerateUniqueName);
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
var logicalDpi = DisplayInformation.GetForCurrentView().LogicalDpi;
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, stream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Ignore,
(uint)render.PixelWidth,
(uint)render.PixelHeight,
logicalDpi,
logicalDpi,
pixel.ToArray());
await encoder.FlushAsync();
stream.Dispose();
MediaClip clip = await MediaClip.CreateFromImageFileAsync(file, TimeSpan.FromSeconds(3));
composition.Clips.Add(clip);
MyText.Text = "First frame >>>" + i;
}
}
var video = await ApplicationData.Current.LocalFolder.CreateFileAsync("test.mp4",
CreationCollisionOption.ReplaceExisting);
var action = await composition.RenderToFileAsync(video, MediaTrimmingPreference.Precise);
await folder.DeleteAsync();
}

Trying To take Screenshot in uwp programatically on tapping of an image

trying to capture screenshot on uwp But some issue is arising,saying
1) writable map doesn't contain a definition for Render.
2) writable map doesn't contain a definition for Save.jpeg
3)Type or namespace media library doesn't found.
any type of help would be appreciated.
Code
private void Download(object sender, TappedRoutedEventArgs e)
{
// Create a WriteableBitmap with height and width same as that of Layoutroot
WriteableBitmap bmp = new WriteableBitmap(480, 696);
//Render the layoutroot element on it
bmp.Render(Scrshot, null);
bmp.Invalidate();
//Save the image to Medialibrary
//create a stream of image
var ms = new MemoryStream();
bmp.SaveJpeg(ms, 480, 696, 0, 100);
ms.Seek(0, SeekOrigin.Begin);
//every path must be unique
var filePath = "myfile" + DateTime.Now.ToString();
//Remember to include Medialib capabilty from WMAppManifest.xml for acessing the Medialibrary
var lib = new MediaLibrary();
lib.SavePicture(filePath, ms);
MessageBox.Show("Saved in your media library!", "Done", MessageBoxButton.OK);
}

Instead of WriteableBitmap use RenderTargetBitmap to create a bitmap representation of a visible UIElement. To save this bitmap as a file you can use this extension method I've created (here's a great example for extension methods):
public static class RenderTargetBitmapExtensions {
public static async Task<StorageFile> ToFile(this RenderTargetBitmap renderTargetBitmap, string filename, StorageFolder folder = null, bool overrideExisting = true) {
if (folder == null) folder = ApplicationData.Current.TemporaryFolder;
try {
byte[] pixels = (await renderTargetBitmap.GetPixelsAsync()).ToArray();
StorageFile outputFile = await folder.CreateFileAsync(filename, overrideExisting ? CreationCollisionOption.ReplaceExisting : CreationCollisionOption.GenerateUnique);
var bitmapEncodingMode = BitmapEncoder.PngEncoderId;
using (var writeStream = await outputFile.OpenAsync(FileAccessMode.ReadWrite)) {
var encoder = await BitmapEncoder.CreateAsync(bitmapEncodingMode, writeStream);
encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Straight, (uint)renderTargetBitmap.PixelWidth, (uint)renderTargetBitmap.PixelHeight, 96, 96, pixels);
await encoder.FlushAsync();
}
return outputFile;
} catch {
return null;
}
}
}
And finally to get the public Images folder (as MediaLibrary is a Silverlight class and does not exist in UWP anymore) you can do the following according to this thread:
StorageFolder picturesLibrary = KnownFolders.PicturesLibrary;
StorageFolder savedPicturesFolder = await picturesLibrary.CreateFolderAsync("Saved Pictures", CreationCollisionOption.OpenIfExists);
Note: By default, apps do not have access to the Pictures Library. You must add the capability in Package.appxmanifest. Open Package.appxmanifest and click on the Capabilities tab. There is a checkbox for the Pictures Library.
So a whole code to do this should be:
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(LayoutRoot, Convert.ToInt32(LayoutRoot.ActualWidth), Convert.ToInt32(LayoutRoot.ActualHeight));
StorageFolder savedPicturesFolder = await KnownFolders.PicturesLibrary.CreateFolderAsync("Saved Pictures", CreationCollisionOption.OpenIfExists);
await renderTargetBitmap.ToFile("filename.jpg", savedPicturesFolder);
Or if you don't want to override existing files, the last line would be:
await renderTargetBitmap.ToFile("filename.jpg", savedPicturesFolder, false);
For that you can alternatively create a time-based filename:
string filename = String.Format("downloaded_{0}.jpg", DateTime.Now.ToString("yyyyMMdd_HHmmss"));
await renderTargetBitmap.ToFile(filename, savedPicturesFolder, false);

Storing a BitmapImage in LocalFolder - UWP

I'm trying to store a BitmapImage to the filesystem using C# on UWP. The image is downloaded from Facebook using the graph api and returned as a BitmapImage. That part works, and to retrieve the image (once I can store it, tested with pictures just dropped in the local folder) I'm using the following code:
public static async Task<BitmapImage> GetProfilePicture(string userId){
BitmapImage profilePicture = new BitmapImage();
StorageFolder pictureFolder = await
ApplicationData.Current.LocalFolder.GetFolderAsync("ProfilePictures");
StorageFile pictureFile = await pictureFolder.GetFileAsync(userId + ".jpg");
IRandomAccessStream stream = await pictureFile.OpenAsync(FileAccessMode.Read);
profilePicture.SetSource(stream);
return profilePicture;
This works as well, so what I would want is to simply do the opposite. The preferred result would look like this:
public static async void SaveBitmapToFile(BitmapImage image, userId){
StorageFolder pictureFolder = await
ApplicationData.Current.LocalFolder.CreateFolderAsync(
"ProfilePictures",CreationCollisionOption.OpenIfExists);
//save bitmap to pictureFolder with name userId.jpg
}
I've searched far and wide trying to find a solution, but I can't seem to find any for the UWP platform. How would a go about saving the Bitmap to file? The extension doesn't have to be .jpg if it would be easier to use another extension.

It would be easier if you used a WriteableBitmap. For example, the first method would then be:
public static async Task<WriteableBitmap> GetProfilePictureAsync(string userId)
{
StorageFolder pictureFolder = await ApplicationData.Current.LocalFolder.GetFolderAsync("ProfilePictures");
StorageFile pictureFile = await pictureFolder.GetFileAsync(userId + ".jpg");
using (IRandomAccessStream stream = await pictureFile .OpenAsync(FileAccessMode.Read))
{
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(stream);
WriteableBitmap bmp = new WriteableBitmap((int)decoder.PixelWidth, (int)decoder.PixelHeight);
await bmp.SetSourceAsync(stream);
return bmp;
}
}
Then you could do:
public static async Task SaveBitmapToFileAsync(WriteableBitmap image, userId)
{
StorageFolder pictureFolder = await ApplicationData.Current.LocalFolder.CreateFolderAsync("ProfilePictures",CreationCollisionOption.OpenIfExists);
var file = await pictureFolder.CreateFileAsync(userId + ".jpg", CreationCollisionOption.ReplaceExisting);
using (var stream = await file.OpenStreamForWriteAsync())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream.AsRandomAccessStream());
var pixelStream = image.PixelBuffer.AsStream();
byte[] pixels = new byte[image.PixelBuffer.Length];
await pixelStream.ReadAsync(pixels, 0, pixels.Length);
encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore, (uint)image.PixelWidth, (uint)image.PixelHeight, 96, 96, pixels);
await encoder.FlushAsync();
}
}

How to save contents of CanvasControl as an image file

I'm making a paint-like program using Win2D. My CanvasControl contains some text, images and some lines which the user has drawn. I want to save the entire contents of this CanvasControl as a file on disk (in any standard image format). I want to do this as I want to display it (at a later time) inside a standard Image control.
How would I do this? I tried using RenderTargetBitmap to load the CanvasControl (code below) but for some reason it clips the image and only a small horizontal top-portion image is made.
async private void Button_Click(object sender, RoutedEventArgs e)
{
#region (c) rendering UIElement to bitmap code
var bitmap = new RenderTargetBitmap();
await bitmap.RenderAsync(ccDraw); // ccDraw is CanvasControl
// get the pixels
IBuffer pixelBuffer = await bitmap.GetPixelsAsync();
byte[] pixels = pixelBuffer.ToArray();
// write the pixels to a InMemoryRandomAccessStream
var stream = new InMemoryRandomAccessStream();
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.BmpEncoderId, stream);
encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Straight, (uint)bitmap.PixelWidth, (uint)bitmap.PixelHeight, 96, 96, pixels);
await encoder.FlushAsync();
stream.Seek(0);
Image iNew = new Image();
iNew.Stretch = Stretch.None;
iNew.Source = bitmap;
gOuter.Children.Add(iNew);
ccDraw.Visibility = Visibility.Collapsed; // hide CanvasControl so we can see added image
#endregion
}

Here is how i did this for me. imageSize is size of your image, for example var imageSize = new Size(500,500);
var displayInformation = DisplayInformation.GetForCurrentView();
ccDraw.Measure(imageSize);
ccDraw.UpdateLayout();
ccDraw.Arrange(new Rect(0, 0, imageSize.Width, imageSize.Height));
var renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(ccDraw, Convert.ToInt32(imageSize.Width), Convert.ToInt32(imageSize.Height));
var pixelBuffer = await renderTargetBitmap.GetPixelsAsync();
var file = await ApplicationData.Current.LocalFolder.CreateFileAsync("Screen.jpg", CreationCollisionOption.ReplaceExisting);
using (var fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, fileStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Ignore,
(uint)renderTargetBitmap.PixelWidth,
(uint)renderTargetBitmap.PixelHeight,
displayInformation.LogicalDpi,
displayInformation.LogicalDpi,
pixelBuffer.ToArray());
await encoder.FlushAsync();
}

Try this:
private void SaveCanvasAsImage()
{
Rect yourRect = new Rect(ccDraw.RenderSize);
RenderTargetBitmap rtBitmap = new RenderTargetBitmap((int)yourRect.Right,
(int)yourRect.Bottom, 100d, 100d, System.Windows.Media.PixelFormats.Default);
rtBitmap.Render(ccDraw);
BitmapEncoder pngEncoder = new PngBitmapEncoder();
pngEncoder.Frames.Add(BitmapFrame.Create(rtBitmap));
System.IO.MemoryStream ms = new System.IO.MemoryStream();
pngEncoder.Save(ms);
ms.Close();
System.IO.File.WriteAllBytes("yourPng.png", ms.ToArray());
}

You cannot directly save the contents of a CanvasControl.
However, if you draw to an intermediate CanvasRenderTarget then you can save that using SaveAsync.
More information on drawing offscreen can be found at http://microsoft.github.io/Win2D/html/Offscreen.htm.
There's other benefits to drawing to a CanvasRenderTarget rather than drawing directly to the CanvasControl - the most obvious is that it allows you to do additive rendering, rather than having to redraw the entire display each time.

Rotating images in WinRT

I have a Windows 8 app in which I want to rotate an image file.
In shot, I want to open an image file, rotate it and save the content back to the file.
Is that possible in WinRT? If so, how? Thanks.
Update:
Base on Vasile's answer, I could do some work on this. However I'm not sure what to do next:
public static async Task RotateImage(StorageFile file)
{
if (file == null)
return;
var data = await FileIO.ReadBufferAsync(file);
// create a stream from the file
var ms = new InMemoryRandomAccessStream();
var dw = new DataWriter(ms);
dw.WriteBuffer(data);
await dw.StoreAsync();
ms.Seek(0);
// find out how big the image is, don't need this if you already know
var bm = new BitmapImage();
await bm.SetSourceAsync(ms);
// create a writable bitmap of the right size
var wb = new WriteableBitmap(bm.PixelWidth, bm.PixelHeight);
ms.Seek(0);
// load the writable bitpamp from the stream
await wb.SetSourceAsync(ms);
wb.Rotate(90);
//How should I save the image to the file now?
}

Ofcourse it is possible. You can do it yourself with a pixel manipulation and create a new WriteableBitmapObject or, you could reuse the already implemented functionality from the WriteableBitmapEx (WriteableBitmap Extensions). You can get it via NuGet.
Here you can find a description of the implemented functionality which it offers, and few short samples.

Use this to save WriteableBitmap to StorageFile
private async Task<StorageFile> WriteableBitmapToStorageFile(WriteableBitmap writeableBitmap)
{
var picker = new FileSavePicker();
picker.FileTypeChoices.Add("JPEG Image", new string[] { ".jpg" });
StorageFile file = await picker.PickSaveFileAsync();
if (file != null && writeableBitmap != null)
{
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(
BitmapEncoder.JpegEncoderId, stream);
Stream pixelStream = writeableBitmap.PixelBuffer.AsStream();
byte[] pixels = new byte[pixelStream.Length];
await pixelStream.ReadAsync(pixels, 0, pixels.Length);
encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore,
(uint)writeableBitmap.PixelWidth, (uint)writeableBitmap.PixelHeight, 96.0, 96.0, pixels);
await encoder.FlushAsync();
}
return file;
}
else
{
return null;
}
}