We Keep Coding

How to crop a software bitmap in rectangular shape - UWP

I'am capturing a frame from running video and converting it to SoftwareBitmap for further purposes. Before that I want to crop that frame into a rectangular shape. How is it possible?
var thumbnail = await GetThumbnailAsync(file,seek_position);
StringBuilder ocr=null;
InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
await RandomAccessStream.CopyAsync(thumbnail, randomAccessStream);
randomAccessStream.Seek(0);
SoftwareBitmap inputBitmap;
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(randomAccessStream);
// Get the SoftwareBitmap representation of the file
inputBitmap = await decoder.GetSoftwareBitmapAsync();
//crop inputBitmap
public async Task<IInputStream> GetThumbnailAsync(StorageFile file,int i)
{
//int duration_millisecond = i * 1000;
var mediaClip = await MediaClip.CreateFromFileAsync(file);
var mediaComposition = new MediaComposition();
mediaComposition.Clips.Add(mediaClip);
return await mediaComposition.GetThumbnailAsync(
TimeSpan.FromMilliseconds(i), 0, 0, VideoFramePrecision.NearestFrame);
}

The GetSoftwareBitmapAsync method of BitmapDecoder object has several overloaded methods. You could use GetSoftwareBitmapAsync(BitmapPixelFormat, BitmapAlphaMode, BitmapTransform, ExifOrientationMode, ColorManagementMode) method to crop the software bitmap. You just need to define a BitmapTransform object for it.
Please refer to the following code sample:
SoftwareBitmap inputBitmap;
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(randomAccessStream);
// Get the SoftwareBitmap representation of the file
inputBitmap = await decoder.GetSoftwareBitmapAsync(decoder.BitmapPixelFormat,BitmapAlphaMode.Ignore,new BitmapTransform() {Bounds=new BitmapBounds() {X=100,Y=200,Width=200,Height=100 } },ExifOrientationMode.IgnoreExifOrientation,ColorManagementMode.DoNotColorManage);
You just need to specify a new BitmapBounds to its Bounds property.
Please note that, at this step, you have got a cropped software bitmap, but if you want to use it to initialize a SoftwareBitmapSource and make it show in Image control. You will get an exception "SetBitmapAsync only supports SoftwareBitmap with positive width/height, bgra8 pixel format and pre-multiplied or no alpha.". You need to use SoftwareBitmap _softbitmap = SoftwareBitmap.Convert() to make a new software bitmap like the following:
SoftwareBitmap _softbitmap = SoftwareBitmap.Convert(inputBitmap,BitmapPixelFormat.Bgra8,BitmapAlphaMode.Premultiplied);
var source = new SoftwareBitmapSource();
await source.SetBitmapAsync(_softbitmap);
image.Source = source; //image is a Image control

How to save media element's current frame?

I am developing a media player like vlc. One of it's features is saving snapshot during playback. I am able to successfully save the snapshot of any element using rendertargetbitmap class, using the following c# code.
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(RenderedGrid, width, height);
RenderedImage.Source = renderTargetBitmap;
But it does not allow to capture media element. Please help me with a workaround. I guess, I am not sure, may it be possible by extracting a thumbnail from a specific position using position of StorageItemThumbnail Class. If even so, it will be a thumbnail not an image of current frame. Someone help please. It is essential feature of my app!

in a digital signage uwp app that I developed I had also a similar requirement. Searching the web on how I can do I found that in UWP you can't directly take a screenshot of a media element due to a protection policy (for example you could create a small app that captures all the frames of a video file without DRM management).
I came up with a workaround: I have the video file locally so I can extract a single frame at a certain point and save it to disk with the same dimensions of the video rendered on screen (in this way I had a full image of the frame to use later). To doing this I wrote this function:
private async Task<string> Capture(StorageFile file, TimeSpan timeOfFrame, Size imageDimension)
{
if (file == null)
{
return null;
}
//Get FrameWidth & FrameHeight
List<string> encodingPropertiesToRetrieve = new List<string>();
encodingPropertiesToRetrieve.Add("System.Video.FrameHeight");
encodingPropertiesToRetrieve.Add("System.Video.FrameWidth");
IDictionary<string, object> encodingProperties = await file.Properties.RetrievePropertiesAsync(encodingPropertiesToRetrieve);
uint frameHeight = (uint)encodingProperties["System.Video.FrameHeight"];
uint frameWidth = (uint)encodingProperties["System.Video.FrameWidth"];
//Get image stream
var clip = await MediaClip.CreateFromFileAsync(file);
var composition = new MediaComposition();
composition.Clips.Add(clip);
var imageStream = await composition.GetThumbnailAsync(timeOfFrame, (int)frameWidth, (int)frameHeight, VideoFramePrecision.NearestFrame);
//Create BMP
var writableBitmap = new WriteableBitmap((int)frameWidth, (int)frameHeight);
writableBitmap.SetSource(imageStream);
//Get stream from BMP
string mediaCaptureFileName = "IMG" + Guid.NewGuid().ToString().Substring(0, 4) + ".jpg";
var saveAsTarget = await CreateMediaFile(mediaCaptureFileName);
Stream stream = writableBitmap.PixelBuffer.AsStream();
byte[] pixels = new byte[(uint)stream.Length];
await stream.ReadAsync(pixels, 0, pixels.Length);
using (var writeStream = await saveAsTarget.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, writeStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)writableBitmap.PixelWidth,
(uint)writableBitmap.PixelHeight,
96,
96,
pixels);
await encoder.FlushAsync();
using (var outputStream = writeStream.GetOutputStreamAt(0))
{
await outputStream.FlushAsync();
}
}
return saveAsTarget.Name;
}
This function save the frame to disk as image and return his filename.
In my application I had more "widget" (some of those with images, some with video) so I used this function to get the video frames and put a new image on top of the media element with the same dimensions and a higher Canvas Z-Index in order to use RenderTargetBitmap for take the final screenshot.
It's not a clean solution but it was a workaround for RenderTargetBitmap not considering MediaElement.
Edit:
I forgot a reference to an internal function in my application so I canghed the previous code and the line var file = await _mediaEngine.CreateMediaFile($"{Guid.NewGuid().ToString()}.jpg"); is now var file = await CreateMediaFile($"{Guid.NewGuid().ToString()}.jpg");
The async function CreateMediaFile(string fileName) simply create a new file on the disk:
public async Task<StorageFile> CreateMediaFile(string filename)
{
StorageFolder _mediaFolder = KnownFolders.PicturesLibrary;
return await _mediaFolder.CreateFileAsync(filename);
}

C# Resize bitmap image before decoder operations

I'm attempting to create a small OCR application where I can select words. I have already implemented this using the Microsoft example:
protected async override void OnNavigatedTo(NavigationEventArgs e)
{
// Load image from install folder.
var file = await Package.Current.InstalledLocation.GetFileAsync("photo.png");
using (var stream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read))
{
// Create image decoder.
var decoder = await BitmapDecoder.CreateAsync(stream);
// Load bitmap.
var bitmap = await decoder.GetSoftwareBitmapAsync();
// Extract text from image.
OcrResult result = await ocrEngine.RecognizeAsync(bitmap);
// Display recognized text.
OcrText.Text = result.Text;
}
}
However, I want the loaded image to fit to the screen, I do this by simply creating a BitmapImage, setting its source to the stream and applying the source of an Image element to the bitmap. This problem with this, is that coords of the OCR words are not aligned to the image, as I presume the bitmap created from the decoder uses the original image size. How can I rectify this issue, so that the size of the image the OCR engine uses is the same as Image element?
My code to get the image, and to get an OCR result from:
// Create a stream from the picked file
IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.Read);
// Create a bitmap from the stream
BitmapImage bitmap = new BitmapImage();
bitmap.SetSource(stream);
this.imgDisplay.Source = bitmap;
// Create the decoder from the stream
var decoder = await BitmapDecoder.CreateAsync(stream);
// Get the SoftwareBitmap representation of the file
SoftwareBitmap sBitmap = await decoder.GetSoftwareBitmapAsync();
mOcrResult = await mOcrEngine.RecognizeAsync(sBitmap);

BitmapImage to byte array and vice versa

I am developing an image processing app in uwp windows 10. I am opening an image using file picker as shown below.
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
openPicker.ViewMode = PickerViewMode.Thumbnail;
openPicker.FileTypeFilter.Clear();
openPicker.FileTypeFilter.Add(".bmp");
openPicker.FileTypeFilter.Add(".png");
openPicker.FileTypeFilter.Add(".jpeg");
openPicker.FileTypeFilter.Add(".jpg");
StorageFile file = await openPicker.PickSingleFileAsync();
if(file!=null)
{
IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.Read);
BitmapImage bitmapImage = new BitmapImage();
bitmapImage.SetSource(fileStream);
myImage.Source = bitmapImage;
// code to retrieve bytes of bitmap image
}
Inside above if statement, I am retrieving bytes from this image like shown below.
//Fetching pixel data
using (IRandomAccessStream fileStreams = await file.OpenAsync(Windows.Storage.FileAccessMode.Read))
{
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(fileStreams);
BitmapTransform transform = new BitmapTransform()
{
ScaledWidth = Convert.ToUInt32(bitmapImage.PixelWidth),
ScaledHeight = Convert.ToUInt32(bitmapImage.PixelHeight)
};
PixelDataProvider pixelData = await decoder.GetPixelDataAsync(
BitmapPixelFormat.Rgba8,
BitmapAlphaMode.Straight,
transform,
ExifOrientationMode.IgnoreExifOrientation,// This sample ignores Exif orientation
ColorManagementMode.DoNotColorManage
);
// byte[] , a global variable
sourcePixels = pixelData.DetachPixelData();
// uint , a global variable
width = decoder.PixelWidth;
// uint , a global variable
height = decoder.PixelHeight;
}
Now I need to manipulate this byte array for generating different effects. But for testing purpose, I am converting this byte array, again to bitmapimage and setting its value to main image source (in button click event). but it is not working correctly as
WriteableBitmap scaledImage = new WriteableBitmap((int)width, (int)height);
using (Stream stream = scaledImage.PixelBuffer.AsStream())
{
await stream.WriteAsync(sourcePixels, 0, sourcePixels.Length);
myImage.Source = scaledImage;
}
when the image was opened, it was like this.
when applied again after changing it to byte array and byte array to image source. It changes the image colors, although i haven't changed any values of byte array.
Where is the problem?? Whether the conversion to byte array is wrong or conversion of byte array to bitmap?

Solution:
Well, I have found the solution of this issue, it was that I was using BitmapPixelFormat.Rgba8 in PixelDataProvider (while fetching pixels data). Rather I should use BitmapPixelFormat.Bgra8.

WriteableBitmap renders PNG incorrectly

I am having trouble rendering PNGs that use Palette as "Color Type". Here is some simple code to reproduce the issue:
private async System.Threading.Tasks.Task Fetch()
{
HttpClient httpClient = new HttpClient();
Uri uri = new Uri("http://static.splashnology.com/articles/How-to-Optimize-PNG-and-JPEG-without-Quality-Loss/PNG-Palette.png");
HttpResponseMessage response = await httpClient.GetAsync(uri);
if (response.StatusCode == HttpStatusCode.Ok)
{
try
{
var content = await response.Content.ReadAsBufferAsync();
WriteableBitmap image = await BitmapFactory.New(1, 1).FromStream(content.AsStream());
Rect destination = new Rect(0, 0, image.PixelWidth, image.PixelHeight);
Rect source = new Rect(0, 0, image.PixelWidth, image.PixelHeight);
WriteableBitmap canvas = new WriteableBitmap(image.PixelWidth, image.PixelHeight);
canvas.Blit(destination, image, source);
RadarImage.Source = canvas;
}
catch (Exception e)
{
System.Diagnostics.Debug.WriteLine(e.Message);
System.Diagnostics.Debug.WriteLine(e.StackTrace);
}
}
}
If I run that code using Windows Phone 8.1, the image appears using wrong colors. If I do the same test using a PNG that is using RGB as "Color Type", then everything is works fine.
I have looked at Codeplex forum and haven't seen any post related to that. I have reported it as an issue although it might be related to the way I'm rendering it. Is there any mistake in the way I'm using WriteableBitmap that could cause the wrong rendering?
UPDATE
According to this discussion
https://writeablebitmapex.codeplex.com/discussions/274445
the issue is related to an unexpected order of the bytes. These comments are from one and a half years ago, so I think there should be a proper fix somewhere...
The image that is rendered incorrectly is in the code above.
This one, using the same code, is rendered correctly.
http://www.queness.com/resources/images/png/apple_ex.png
The only difference between these two images is the "Color Type" property. The one that fails is set to "Palette" and the one rendered correctly is set to "RGB Alpha".
Thanks!
Carlos.

The problem appears to be in the FromStream extension, which seems to translate the paletted png to RGBA. As you note, WriteableBitmap wants BGRA. I suspect FromStream passes non-palette pngs' pixels unswizzled. This lets the apple start and finish as BGRA while the monkey ends up RGBA and draws with red and blue reversed.
You can bypass this problem by skipping FromStream and using a BitmapDecoder so you specify the format you want it to decode into:
// Read the retrieved image's bytes and write them into an IRandomAccessStream
IBuffer buffer = await response.Content.ReadAsBufferAsync();
var randomAccessStream = new InMemoryRandomAccessStream();
await randomAccessStream.WriteAsync(buffer);
// Decode the downloaded image as Bgra8 with premultiplied alpha
// GetPixelDataAsync lets us pass in the desired format and it'll do the magic to translate as it decodes
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(randomAccessStream);
var pixelData = await decoder.GetPixelDataAsync(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied, new BitmapTransform(), ExifOrientationMode.IgnoreExifOrientation, ColorManagementMode.DoNotColorManage);
// Get the decoded bytes
byte[] imageData = pixelData.DetachPixelData();
// And stick them in a WriteableBitmap
WriteableBitmap image = new WriteableBitmap((int)decoder.PixelWidth,(int) decoder.PixelHeight);
Stream pixelStream = image.PixelBuffer.AsStream();
pixelStream.Seek(0, SeekOrigin.Begin);
pixelStream.Write(imageData, 0, imageData.Length);
// And stick it in an Image so we can see it.
RadarImage.Source = image;