We Keep Coding

How to downscale the pixel values in UWP App (C#)?

I built a UWP App based on the Squeeze Net example provided in the repository (C#) that uses a Deep Learning model (ONNX) for image classification. I have built the deep learning model in PyTorch where the pixel values of the image have been scaled down from the range [0, 255] to [0, 1] and then normalized with channel wise (RGB) standard deviation and mean. So, this model expects the pixel values other than [0, 255] range.
But in the UWP App, I'm unable to perform this downscaling of the pixel values before binding the inputs to the model. I have searched the SoftwareBitmap class but couldn't find a way to perform this downscaling operation. Any help would be very very appreciated.
I need this operation somewhere in between these lines of code.
await LoadModel();
// Trigger file picker to select an image file
var picker = new FileOpenPicker();
picker.ViewMode = PickerViewMode.Thumbnail;
picker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
picker.FileTypeFilter.Add(".jpg");
picker.FileTypeFilter.Add(".jpeg");
picker.FileTypeFilter.Add(".png");
StorageFile file = await picker.PickSingleFileAsync();
outputTextBlock.Text = $"The selected Image: {file.Name}";
SoftwareBitmap softwareBitmap;
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.Read))
{
// Create the decoder from the stream
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(stream);
PixelDataProvider random = await decoder.GetPixelDataAsync();
// byte[] pD = random.DetachPixelData();
//await FileIO.WriteBytesAsync("path/file.ext", pD);
// System.IO.File.WriteAllBytes("path/file.ext", pD);
// byteData.Text = $"{pD}";
// Get the SoftwareBitmap representation of the file in BGRA8 format
softwareBitmap = await decoder.GetSoftwareBitmapAsync();
softwareBitmap = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore);
}
var streamD = await file.OpenReadAsync();
var imageSource = new BitmapImage();
await imageSource.SetSourceAsync(streamD);
selectedImage.Source = imageSource;
// Display the image
//SoftwareBitmapSource imageSource = new SoftwareBitmapSource();
//await imageSource.SetBitmapAsync(softwareBitmap);
//selectedImage.Source = imageSource;
// Encapsulate the image within a VideoFrame to be bound and evaluated
VideoFrame inputWoodImage = VideoFrame.CreateWithSoftwareBitmap(softwareBitmap);
await EvaluateVideoFrameAsync(inputWoodImage);

this is a repost of this thread here that contains some answers and is more actual:
https://github.com/Microsoft/Windows-Machine-Learning/issues/22

How to crop a software bitmap in rectangular shape - UWP

I'am capturing a frame from running video and converting it to SoftwareBitmap for further purposes. Before that I want to crop that frame into a rectangular shape. How is it possible?
var thumbnail = await GetThumbnailAsync(file,seek_position);
StringBuilder ocr=null;
InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
await RandomAccessStream.CopyAsync(thumbnail, randomAccessStream);
randomAccessStream.Seek(0);
SoftwareBitmap inputBitmap;
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(randomAccessStream);
// Get the SoftwareBitmap representation of the file
inputBitmap = await decoder.GetSoftwareBitmapAsync();
//crop inputBitmap
public async Task<IInputStream> GetThumbnailAsync(StorageFile file,int i)
{
//int duration_millisecond = i * 1000;
var mediaClip = await MediaClip.CreateFromFileAsync(file);
var mediaComposition = new MediaComposition();
mediaComposition.Clips.Add(mediaClip);
return await mediaComposition.GetThumbnailAsync(
TimeSpan.FromMilliseconds(i), 0, 0, VideoFramePrecision.NearestFrame);
}

The GetSoftwareBitmapAsync method of BitmapDecoder object has several overloaded methods. You could use GetSoftwareBitmapAsync(BitmapPixelFormat, BitmapAlphaMode, BitmapTransform, ExifOrientationMode, ColorManagementMode) method to crop the software bitmap. You just need to define a BitmapTransform object for it.
Please refer to the following code sample:
SoftwareBitmap inputBitmap;
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(randomAccessStream);
// Get the SoftwareBitmap representation of the file
inputBitmap = await decoder.GetSoftwareBitmapAsync(decoder.BitmapPixelFormat,BitmapAlphaMode.Ignore,new BitmapTransform() {Bounds=new BitmapBounds() {X=100,Y=200,Width=200,Height=100 } },ExifOrientationMode.IgnoreExifOrientation,ColorManagementMode.DoNotColorManage);
You just need to specify a new BitmapBounds to its Bounds property.
Please note that, at this step, you have got a cropped software bitmap, but if you want to use it to initialize a SoftwareBitmapSource and make it show in Image control. You will get an exception "SetBitmapAsync only supports SoftwareBitmap with positive width/height, bgra8 pixel format and pre-multiplied or no alpha.". You need to use SoftwareBitmap _softbitmap = SoftwareBitmap.Convert() to make a new software bitmap like the following:
SoftwareBitmap _softbitmap = SoftwareBitmap.Convert(inputBitmap,BitmapPixelFormat.Bgra8,BitmapAlphaMode.Premultiplied);
var source = new SoftwareBitmapSource();
await source.SetBitmapAsync(_softbitmap);
image.Source = source; //image is a Image control

C# Resize bitmap image before decoder operations

I'm attempting to create a small OCR application where I can select words. I have already implemented this using the Microsoft example:
protected async override void OnNavigatedTo(NavigationEventArgs e)
{
// Load image from install folder.
var file = await Package.Current.InstalledLocation.GetFileAsync("photo.png");
using (var stream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read))
{
// Create image decoder.
var decoder = await BitmapDecoder.CreateAsync(stream);
// Load bitmap.
var bitmap = await decoder.GetSoftwareBitmapAsync();
// Extract text from image.
OcrResult result = await ocrEngine.RecognizeAsync(bitmap);
// Display recognized text.
OcrText.Text = result.Text;
}
}
However, I want the loaded image to fit to the screen, I do this by simply creating a BitmapImage, setting its source to the stream and applying the source of an Image element to the bitmap. This problem with this, is that coords of the OCR words are not aligned to the image, as I presume the bitmap created from the decoder uses the original image size. How can I rectify this issue, so that the size of the image the OCR engine uses is the same as Image element?
My code to get the image, and to get an OCR result from:
// Create a stream from the picked file
IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.Read);
// Create a bitmap from the stream
BitmapImage bitmap = new BitmapImage();
bitmap.SetSource(stream);
this.imgDisplay.Source = bitmap;
// Create the decoder from the stream
var decoder = await BitmapDecoder.CreateAsync(stream);
// Get the SoftwareBitmap representation of the file
SoftwareBitmap sBitmap = await decoder.GetSoftwareBitmapAsync();
mOcrResult = await mOcrEngine.RecognizeAsync(sBitmap);

Windows Store - Pixels Array

I am working at an app where I need to get the array of pixels from an Image and to edit the Image using the pixels array.
I am using the next code for getting the pixels array from the StorageFile object which indicates the image:
public static async Task<byte[]> GetPixelsArrayFromStorageFileAsync(
IRandomAccessStreamReference file)
{
using (IRandomAccessStream stream = await file.OpenReadAsync())
{
using (var reader = new DataReader(stream.GetInputStreamAt(0)))
{
await reader.LoadAsync((uint)stream.Size);
var pixelByte = new byte[stream.Size];
reader.ReadBytes(pixelByte);
return pixelByte;
}
}
}
Now, my questions are:
Why if I load a image which is 6000 x 4000 pixels I have an array of just 8,941,799 which is actually the size of my image on disk?
How can I access the RGBA channels of the pixels?

Your file has a compressed version of the bitmap, so you need to decode it first. I'd suggest loading it into a WriteableBitmap since you need to display it anyway and then access the PixelBuffer property of the bitmap to get the actual pixels. You could do something like this:
var writeableBitmap = new WriteableBitmap(1, 1);
await writeableBitmap.SetSourceAsync(yourFileStream);
var pixelStream = writeableBitmap.PixelBuffer.AsStream();
var bytes = new byte[pixelStream.Length];
pixelStream.Seek(0, SeekOrigin.Begin);
pixelStream.Read(bytes, 0, Bytes.Length);
// Update the bytes here. I think they follow the BGRA pixel format.
pixelStream.Seek(0, SeekOrigin.Begin);
pixelStream.Write(bytes, 0, bytes.Length);
writeableBitmap.Invalidate();
You can check the extension methods here and here to see how to work with the pixels.

Silverlight 4 : Converting image into byte[]

I have found how to do this in .NET 4.0, but I think JpegBitmapEncoder doesn't exist in Silverlight:
MemoryStream memStream = new MemoryStream();
JpegBitmapEncoder encoder = new JpegBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(imageC));
encoder.Save(memStream);
var bytes = memStream.GetBuffer();
How can I convert an image to bytes[] in silverlight?
UPDATE:
I have a Contact model, which has a Photo property. Whenever I add a new Contact, I would like to load a local default Image and convert it and set the Photo property to it.
var bitmapImage = new BitmapImage
{
UriSource = new Uri("pack://application:,,,/xxx;component/Images/default.JPG")
};
var image = new Image{Source = bitmapImage};
Is this the correct way to load an image in first place?

Use
myImage.Save(memStream, ImageFormat.Jpeg);
return memStream.ToArray();
UPDATE
OK it turns out that the image is a BitmapImage.
It seems that BitmapImage does not expose the functionality to save the image. The solution is to get the image from the embedded resource:
Stream s = Assembly.GetExecutingAssembly().GetManifestResourceStream(resourcePath);
byte[] buffer = new byte[s.Length];
s.Read(buffer, 0, buffer.Length);

Have a look at this library: Imagetools
It contains some nice utilities and jpg and png encoders,