We Keep Coding

Send InkCanvas signature using WCF call

I have a Windows 10 UWP app that will run on Windows 10 Mobile. A requirement I have is to capture a signature from the user. So far, I am simply using an InkCanvas in XAML and have it wired up to my code behind.
I then have a button that when clicked, will take the signature on the InkCanvas and send it to the server via a WCF call. The server and WCF service is already existing. It takes in the signature image as a base64 serialized string.
I know how to get the base64 once I have either an image or a byte array. However, in my many hours of reading, I am finding that articles/examples were either written for WPF or Windows 8.1 and do not work on Windows 10 UWP. Also, of the examples that I have found that will work, it seems my only option is to save the signature to file as a GIF.
I see that I can call GetStrokes() like this
var strokeCollection = cvsSignature.InkPresenter.StrokeContainer.GetStrokes();
Which will return me a read only list of InkStroke. I guess I can iterate that list and build a byte array? How would I do that? It seems this is not efficient?
Otherwise, I thought I could just change the stream from a file stream to a memory stream but I guess either this is not possible or I am missing something. I am trying this
using (var inkMemStream = new MemoryStream())
{
await cvsSignature.InkPresenter.StrokeContainer.SaveAsync(inkMemStream);
}
But with this type of approach I get an exception that I cannot convert from System.IO.MemoryStream to Windows.Storage.Streams.IOutputStream
Thanks!

I found a way, if you don't want to save your InkCanvas to a file or as a GIF, to manipulate it in memory. I actually have three options below. Note, for some of this, you will need to add a reference to Win2D.uwp, which can be found on NuGet by searching that exact name. It is provided by Microsoft.
Convert the InkCanvas to a byte array:
private byte[] ConvertInkCanvasToByteArray()
{
//First, we need to get all of the strokes in the canvas
var canvasStrokes = myCanvas.InkPresenter.StrokeContainer.GetStrokes();
//Just as a check, make sure to only do work if there are actually strokes (ie not empty)
if (canvasStrokes.Count > 0)
{
var width = (int)myCanvas.ActualWidth;
var height = (int)myCanvas.ActualHeight;
var device = CanvasDevice.GetSharedDevice();
//Create a new renderTarget with the same width and height as myCanvas at 96dpi
var renderTarget = new CanvasRenderTarget(device, width,
height, 96);
using (var ds = renderTarget.CreateDrawingSession())
{
//This will clear the renderTarget with a clean slate of a white background.
ds.Clear(Windows.UI.Colors.White);
//Here is where we actually take the strokes from the canvas and draw them on the render target.
ds.DrawInk(myCanvas.InkPresenter.StrokeContainer.GetStrokes());
}
//Finally, this will return the render target as a byte array.
return renderTarget.GetPixelBytes();
}
else
{
return null;
}
}
If all you need is a byte array of the pixels, you are done. However, if you now want to generate an image, you use the WriteableBitmap to do so. Here are sync and async methods that can be called to do this.
private WriteableBitmap GetSignatureBitmapFull()
{
var bytes = ConvertInkCanvasToByteArray();
if (bytes != null)
{
var width = (int)cvsSignature.ActualWidth;
var height = (int)cvsSignature.ActualHeight;
var bmp = new WriteableBitmap(width, height);
using (var stream = bmp.PixelBuffer.AsStream())
{
stream.Write(bytes, 0, bytes.Length);
return bmp;
}
}
else
return null;
}
private async Task<WriteableBitmap> GetSignatureBitmapFullAsync()
{
var bytes = ConvertInkCanvasToByteArray();
if (bytes != null)
{
var width = (int)cvsSignature.ActualWidth;
var height = (int)cvsSignature.ActualHeight;
var bmp = new WriteableBitmap(width, height);
using (var stream = bmp.PixelBuffer.AsStream())
{
await stream.WriteAsync(bytes, 0, bytes.Length);
return bmp;
}
}
else
return null;
}
Finally, here is an async method that I am using to be able to do a crop on the bitmap if you want to do that. The key thing with this method is notice how with this approach, you don't have to get the pixel bytes as an array first. You just get the strokes from the canvas and then it is saved directly into a memory stream
private async Task<WriteableBitmap> GetSignatureBitmapCropped()
{
try
{
var canvasStrokes = cvsSignature.InkPresenter.StrokeContainer.GetStrokes();
if (canvasStrokes.Count > 0)
{
var bounds = cvsSignature.InkPresenter.StrokeContainer.BoundingRect;
var xOffset = (uint)Math.Round(bounds.X);
var yOffset = (uint)Math.Round(bounds.Y);
var pixelWidth = (int)Math.Round(bounds.Width);
var pixelHeight = (int)Math.Round(bounds.Height);
using (var memStream = new InMemoryRandomAccessStream())
{
await cvsSignature.InkPresenter.StrokeContainer.SaveAsync(memStream);
var decoder = await BitmapDecoder.CreateAsync(memStream);
var transform = new BitmapTransform();
var newBounds = new BitmapBounds();
newBounds.X = 0;
newBounds.Y = 0;
newBounds.Width = (uint)pixelWidth;
newBounds.Height = (uint)pixelHeight;
transform.Bounds = newBounds;
var pdp = await decoder.GetPixelDataAsync(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Straight,
transform,
ExifOrientationMode.IgnoreExifOrientation,
ColorManagementMode.DoNotColorManage);
var pixels = pdp.DetachPixelData();
var cropBmp = new WriteableBitmap(pixelWidth, pixelHeight);
using (var stream = cropBmp.PixelBuffer.AsStream())
{
await stream.WriteAsync(pixels, 0, pixels.Length);
}
return cropBmp;
}
}
else
{
return null;
}
}
catch (Exception ex)
{
return null;
}
}
Hope this helps provide some alternatives when using InkCanvas in Windows 10 Universal.

it seems my only option is to save the signature to file as a GIF.
GIF is not the only choice. The official sample just show GIF, but you can also save the InkStokes collection to JPG and PNG. Code as follows:
var savePicker = new Windows.Storage.Pickers.FileSavePicker();
savePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.PicturesLibrary;
savePicker.FileTypeChoices.Add("Gif,JPG,PNG", new System.Collections.Generic.List<string> { ".jpg" ,".gif",".png"});
Windows.Storage.StorageFile file = await savePicker.PickSaveFileAsync();
But with this type of approach I get an exception that I cannot convert from System.IO.MemoryStream to Windows.Storage.Streams.IOutputStream
As you see, it cannot convert from System.IO.MemoryStream to Windows.Storage.Streams.IOutputStream directly. You need to read the image file you just saved as ImageSource. As you known how to get the base64 once you have either an image or a byte array. So what we just need is to get an image or a byte array from the image file.
Read the image file as byte array and then a memory stream as follows
if (inkCanvas.InkPresenter.StrokeContainer.GetStrokes().Count > 0)
{
var savePicker = new Windows.Storage.Pickers.FileSavePicker();
savePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.PicturesLibrary;
savePicker.FileTypeChoices.Add("Gif,JPG,PNG", new System.Collections.Generic.List<string> { ".jpg", ".gif", ".png" });
Windows.Storage.StorageFile file = await savePicker.PickSaveFileAsync();
if (null != file)
{
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
await inkCanvas.InkPresenter.StrokeContainer.SaveAsync(stream);
}
using (IRandomAccessStream streamforread = await file.OpenAsync(FileAccessMode.Read))
{
WriteableBitmap inkimagesource = new WriteableBitmap(50, 50);
inkimagesource.SetSource(streamforread);
byte[] imageBuffer = inkimagesource.PixelBuffer.ToArray();
MemoryStream ms = new MemoryStream(imageBuffer);
}
}
}
Read the file as image source just need to change the file reading code as follows:
using (IRandomAccessStream streamforread = await file.OpenAsync(FileAccessMode.Read)
{
var bitmap = new BitmapImage();
bitmap.SetSource(streamforread);
}
I guess I can iterate that list and build a byte array? How would I do that? It seems this is not efficient?
It seems like currently there is no API can build the InkStroke collections as byte array directly. SaveAsync() method has already help you save the InkStroke collection to a stream, you can use code above for using it.

How to convert RednerTargetBitmap image into byteArray for uploading on server in UWP

I have a grid that covers whole screen and i want to make that grid as image and then this image i want to send on server. I have used RenderTargetBitmap for this purpose and successfully make writeablebitmap to save using FileSave Picker. Image save is at normal size as expected 700kb but these bytes are too big for uploading on server. Expected bytes for 700kb image could be 200 thousand but in my situation bytes are more than 30 hundred thousand. There is definitely issue with bytes.
Here is code that i use to save grid as image using file save picker.
await bitmap.RenderAsync(testgrid);
var pixelBuffer = await bitmap.GetPixelsAsync();
byte[] pixels = pixelBuffer.ToArray();
var wb = new WriteableBitmap((int)bitmap.PixelWidth, (int)bitmap.PixelHeight);
using (stream2 = wb.PixelBuffer.AsStream())
{
await stream2.WriteAsync(pixels, 0, pixels.Length);
}
FileSavePicker picker = new FileSavePicker();
picker.FileTypeChoices.Add("JPG File", new List<string>() { ".jpg" });
StorageFile file = await picker.PickSaveFileAsync();
if (file != null)
{
await (wb as WriteableBitmap).SaveAsync(file);
}
The above code successfully save image to given location with normal size.
but when i use above pixel byte array to server. Http send task cancelled exception while send request and i have detail checked on it. It is due to huge amount of Bytes array.
Also if i send a very small grid of 50x50 then uploading done successfully because it has 30 thousand bytes only but image uploaded on the server is empty or corrupted.
Also i have used above converted writeablebitmap to make its bytes array using this method...
using (Stream stream = mywriteablebitmap.PixelBuffer.AsStream())
using (MemoryStream memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
return memoryStream.ToArray();
}
it also returns same number of bytes Array and same error occurred from server.
Please tell me the correct way of making byte array from RenderTargetBitmap and that can easily be uploaded on server.

but when i use above pixel byte array to server. Http send task cancelled exception while send request and i have detail checked on it. It is due to huge amount of Bytes array.
I totally agree with Clemens,you need to encode your image before uploading it. You can use the following codes to encode your image:
private async Task<String> ToBase64(byte[] image, uint height, uint width, double dpiX = 96, double dpiY = 96)
{
var encoded = new InMemoryRandomAccessStream();
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, encoded);
encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Straight, height, width, dpiX, dpiY, image);
await encoder.FlushAsync();
encoded.Seek(0);
var bytes = new byte[encoded.Size];
await encoded.AsStream().ReadAsync(bytes, 0, bytes.Length);
var base64String=Convert.ToBase64String(bytes);
return base64String;
}
And call it in your codes:
var pixelBuffer = await bitmap.GetPixelsAsync();
byte[] pixels = pixelBuffer.ToArray();
await ToBase64(pixels, (uint)bitmap.PixelHeight, (uint)bitmap.PixelWidth);
For details about encoding Image in WinRT you can refer to Reading and Writing Base64 in the Windows Runtime.

After lot of searching i save the renderTargetBitmap in to isolatedstorage and then access the image file from storage and make its stream byte array using this method.
var stream = await imageStorageFile.OpenStreamForReadAsync();
imageBytes=ReadFully(stream);
public byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
now this method works perfect.

Best way to send images between an Universal Application and a web service

Hi i'm making an aplication that takes a picture from a car and its driver and this pictures are send to a web service, i am having problems for dealing with this situation, right now i've tried to send an encoded 64 base string, but this is not working
EDIT
Here is the code that i'm using for take the picture and save it as a byte array
ImageEncodingProperties format = ImageEncodingProperties.CreateJpeg();
//rotate and save the image
using (var imageStream = new InMemoryRandomAccessStream())
{
//generate stream from MediaCapture
await PhotoCapture.CapturePhotoToStreamAsync(format, imageStream);
await PhotoCapture.StopPreviewAsync();
//create decoder and encoder
BitmapDecoder dec = await BitmapDecoder.CreateAsync(imageStream);
BitmapEncoder enc = await BitmapEncoder.CreateForTranscodingAsync(imageStream, dec);
//roate the image
enc.BitmapTransform.Rotation = BitmapRotation.Clockwise90Degrees;
//write changes to the image stream
await enc.FlushAsync();
PreviewImage = new WriteableBitmap(350, 500);
PreviewImage.SetSource(imageStream);
}
CaptureElement.Visibility = Windows.UI.Xaml.Visibility.Collapsed;
using (var ms = PreviewImage.PixelBuffer.AsStream())
{
MemoryStream memoryStream = new MemoryStream();
ms.CopyTo(memoryStream);
DriverModel.Picture = memoryStream.ToArray();
}
When i'm doing this the array size is like 28 millions, obviusly i'm making something wrong

OutOfMemoryException occurs on WriteableBitmap

I am developing one windows phone application which useful for upload images to web server. I am selecting all images from my device into one List object. I am converting all bitmap image to byte[] one by one.
My code
public byte[] ConvertToBytes(BitmapImage bitmapImage)
{
byte[] data = null;
WriteableBitmap wBitmap = null;
using (MemoryStream stream = new MemoryStream())
{
wBitmap = new WriteableBitmap(bitmapImage);
wBitmap.SaveJpeg(stream, wBitmap.PixelWidth, wBitmap.PixelHeight, 0, 100);
stream.Seek(0, SeekOrigin.Begin);
//data = stream.GetBuffer();
data = stream.ToArray();
DisposeImage(bitmapImage);
return data;
}
}
public void DisposeImage(BitmapImage image)
{
if (image != null)
{
try
{
using (MemoryStream ms = new MemoryStream(new byte[] { 0x0 }))
{
image.SetSource(ms);
}
}
catch (Exception ex)
{
}
}
}
Conversion from bitmap to byte
using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
{
if (!store.DirectoryExists("ImagesZipFolder"))
{
//MediaImage mediaImage = new MediaImage();
//mediaImage.ImageFile = decodeImage(new byte[imgStream[0].Length]);
//lstImages.Items.Add(mediaImage);
store.CreateDirectory("ImagesZipFolder");
for (int i = 0; i < imgname.Count(); i++)
{
using (IsolatedStorageFileStream stream = new IsolatedStorageFileStream(#"ImagesZipFolder\" + imgname[i], FileMode.CreateNew,store))
//using (IsolatedStorageFileStream stream = new IsolatedStorageFileStream(#"ImagesZipFolder\text.txt" , System.IO.FileMode.OpenOrCreate, store))
{
// byte[] bytes = new byte[imgStream[i].Length];
byte[] bytes = ConvertToBytes(ImgCollection[i]);
stream.Write(bytes, 0, bytes.Length);
}
}
}
else {
directory = true;
}
}
I have 91 images in my emulator. When I am converting all these bitmap images into byte[], get's following error on line wBitmap = new WriteableBitmap(bitmapImage);
An exception of type 'System.OutOfMemoryException' occurred in System.Windows.ni.dll but was not handled in user code
What can I do to solve this error?
Web Service
If we are sending huge file to web service, is it gives error like
An exception of type 'System.OutOfMemoryException' occurred in System.ServiceModel.ni.dll but was not handled in user code

What can I do to solve this error?
Change the way you do things, to use less memory.
For instance, instead of converting every picture then uploading them, convert a picture, upload it, convert the next, and so on.
If you really want to process all the pictures at once, you can store the byte arrays in the isolated storage rather than keeping them in memory, and read them back when needed.
Basically, rethink your process and use the storage to use less memory at a given time.
That or ask Microsoft to lift the memory limit on Windows Phone, but it may be a tad trickier.

The problem lies within how the GC and bitmap images work together as described in this bug report: https://connect.microsoft.com/VisualStudio/feedback/details/679802/catastrophic-failure-exception-thrown-after-loading-too-many-bitmapimage-objects-from-a-stream#details
From the report:
When Silverlight loads an image, the framework keeps a reference and
caches the decoded image until flow control is returned to the UI
thread dispatcher. When you load images in a tight loop like that,
even though your application doesn't retain a reference, the GC can't
free the image until we release our reference when flow control is
returned.
After processing 20 or so images, you could stop and queue the next
set using Dispatcher.BeginInvoke just to break up the work that is
processed in one batch. This will allow us to free images that aren't
retained by your application.
I understand with the current decode behavior it's not obvious that
Silverlight is retaining these references, but changing the decoder
design could impact other areas, so for now I recommend processing
images like this in batches.
Now, if you're actually trying to load 500 images and retain them, you
are still likely to run out of memory depending on image size. If
you're dealing with a multi-page document, you may want to instead
load pages on demand in the background and release them when out of
view with a few pages of buffer so that at no point do you exceed
reasonable texture memory limits.
Fix:
private List<BitmapImage> Images = .....;
private List<BitmapImage>.Enumerator iterator;
private List<byte[]> bytesData = new List<byte[]>();
public void ProcessImages()
{
if(iterator == null)
iterator = Images.GetEnumerator();
if(iterator.MoveNext())
{
bytesData.Add(ConvertToBytes(iterator.Current));
//load next images
Dispatcher.BeginInvoke(() => ProcessImages());
}else{
//all images done
}
}
public static byte[] ConvertToBytes(BitmapImage bitmapImage)
{
using (MemoryStream stream = new MemoryStream())
{
var wBitmap = new WriteableBitmap(bitmapImage);
wBitmap.SaveJpeg(stream, wBitmap.PixelWidth, wBitmap.PixelHeight, 0, 100);
stream.Seek(0, SeekOrigin.Begin);
return stream.ToArray();
}
}

I have found new way to convert Bitmap Image to byte[] array. There is no need to write image in memory.
Hare is code
public byte[] GetBytes(BitmapImage bi)
{
WriteableBitmap wbm = new WriteableBitmap(bi);
return ToByteArray(wbm);
}
public byte[] ToByteArray(WriteableBitmap bmp)
{
// Init buffer
int w = bmp.PixelWidth;
int h = bmp.PixelHeight;
int[] p = bmp.Pixels;
int len = p.Length;
byte[] result = new byte[4 * w * h];
// Copy pixels to buffer
for (int i = 0, j = 0; i < len; i++, j += 4)
{
int color = p[i];
result[j + 0] = (byte)(color >> 24); // A
result[j + 1] = (byte)(color >> 16); // R
result[j + 2] = (byte)(color >> 8); // G
result[j + 3] = (byte)(color); // B
}
return result;
}

IsolatedStorage causes Memory to run out

hey.
I'm reading an image from Isolated Storage when the user clicks on an item like this:
using (IsolatedStorageFile currentIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (var img = currentIsolatedStorage.OpenFile(fileName, FileMode.Open))
{
byte[] buffer = new byte[img.Length];
imgStream = new MemoryStream(buffer);
//read the imagestream into the byte array
int read;
while ((read = img.Read(buffer, 0, buffer.Length)) > 0)
{
img.Write(buffer, 0, read);
}
img.Close();
}
}
This works fine, but if I click back and forth between two images, the memory consumption keeps increasing and then runs out of memory. Is there a more efficient way of reading images from Isolated Storage? I could cache a few images in memory, but with hundreds of results, it ends up taking up memory anyway. Any suggestions?

Are you disposing the MemoryStream at some point? This is the only leak I could find.
Also, Stream has a CopyTo() method. Your code could be rewritten like:
using (IsolatedStorageFile currentIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (var img = currentIsolatedStorage.OpenFile(fileName, FileMode.Open))
{
var imgStream = new MemoryStream(img.Length);
img.CopyTo(imgStream);
return imgStream;
}
}
This will save many many memory allocations.
EDIT:
And for Windows Phone (which does not define a CopyTo()), replaced the CopyTo() method with it's code:
using (IsolatedStorageFile currentIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (var img = currentIsolatedStorage.OpenFile(fileName, FileMode.Open))
{
var imgStream = new MemoryStream(img.Length);
var buffer = new byte[Math.Min(1024, img.Length)];
int read;
while ((read = img.Read(buffer, 0, buffer.Length)) != 0)
imgStream.Write(buffer, 0, read);
return imgStream;
}
}
The main difference here is that the buffer is set relatively small (1K). Also, added an optimization by providing the constructor of MemoryStream with the length of the image. That makes MemoryStream pre-alloc the necessary space.