We Keep Coding

Kinect Depth and IR image in UWP (remove color)

I am working with Kinect V2 UWP C# for XBOX and windows. I followed Kinect UWP demo for this purpose. I was able to read and display frames as also shown in Camera Frame sample but I noticed that the Depth and IR images are in color for example:
Kinect studio and UWP application output
I am new this and have tried to search but not found a clear answer. Can anyone help please?
I would really appreciate this.

I tried many things and finally found a way to get images without pseudo colors.
Since XAML only displays in format Bgra8, it needed to be converted. It helped processing frames separately for color and depth as well.
I also needed to update my Windows 10 version to 10.0.19041.0 or later.
//clrFrame.
var buffFrame = clrFrame?.BufferMediaFrame;
// Get the Individual Frame
var vidFrame = clrFrame?.VideoMediaFrame;
{
if (vidFrame == null) return;
// create a UWP SoftwareBitmap and copy Frame into Bitmap
SoftwareBitmap sbt = new SoftwareBitmap(vidFrame.SoftwareBitmap.BitmapPixelFormat, vidFrame.SoftwareBitmap.PixelWidth, vidFrame.SoftwareBitmap.PixelHeight);
vidFrame.SoftwareBitmap.CopyTo(sbt);
// PixelFormat needs to be in 8bit BGRA for Xaml writable bitmap
if (sbt.BitmapPixelFormat != BitmapPixelFormat.Bgra8)
sbt = SoftwareBitmap.Convert(vidFrame.SoftwareBitmap, BitmapPixelFormat.Bgra8);
if (source != null)
{
// To write out to writable bitmap which will be used with ImageElement, it needs to run
// on UI Thread thus we use Dispatcher.RunAsync()...
var ignore = Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
// This code runs on UI Thread
// Create the writableBitmap for ImageElement display
extBitmap = new WriteableBitmap(sbt.PixelWidth, sbt.PixelHeight);
// Copy contents from UWP software Bitmap
// There are other ways of doing this instead of the double copy, 1st copy earlier
// this is a second copy.
sbt.CopyToBuffer(extBitmap.PixelBuffer);
extBitmap.Invalidate();
// Set the imageElement source
var ig = source.SetBitmapAsync(sbt);
imgView.Source = source;
});
}
}
The following project sample helps with this problem. I had to create processing for IR and depth and pass appropriate parameters.
https://github.com/dngoins/KinectUWPApps/tree/master/WorkingWithMediaCaptureFramesSolution

C# Bitmap and Graphic, memory issues and storing screenshots in memory

To start off with, to give a clear picture.
On top of my class I have:
Color FocusArea;
Bitmap FocusImage = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
and a small function to make screenshots
private void Screenshot()
{
Bitmap screenshot = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
Graphics g = Graphics.FromImage(screenshot);
g.CopyFromScreen(startscanx, startscany, 0, 0, scanarea, CopyPixelOperation.SourceCopy);
g.Dispose();
FocusImage = screenshot;
//screenshot.Dispose(); // Shit happens when Disposing this one, wtf ?
}
After a minute or so (+- 3 to 6 screenshots per second) the memory has build up to around 4 to 10gb and crashes obviously.
the line it points out when crashing is :
Bitmap screenshot = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
I tried to dispose it after putting screenshot in FocusImage, but then it crashes right away when using ForusImage in my application, I guess I am doing something horrible wrong here, I just cannot figure out what yet, I hope you can help me pointing out what I'm doing wrong?

The answer came from a comment of GSerg.
Before reassigning the value to FocusImage i had to Dispose it first, this seems to fix the issue, memory is nice and stable at a fixed value.
[Edit] Because someone was so triggered to downvote my question, i got the feeling i know who and why, so here is the response to it:
There were no error messages, it crashed and that was it.
0 information from visual studio in this regard, i am assuming it was because i ran out of memory at that stage.

InkPicture control - How to save both ink and image to an image file

I've seen articles demonstrating that the ink can be saved with the background image so that the ink is overlayed onto the image and then saved as a single image file. But, unfortunately the articles I've seen don't go into any detail about how it is done.
I can save the background image easy enough with the following statement:
axInkPicture1.Picture.Save(#"\path\to\file.gif");
...but there is no ink overlayed onto the background image
I don't know of any direct method to save the ink, but this is how I am currently doing it:
string tempPath = Path.GetTempPath();
string tempFile = tempPath + #"\file.gif";
// save to a temp file
if (axInkPicture1.Ink.Strokes.Count > 0)
{
byte[] bytes = (byte[])axInkPicture1.Ink.Save(InkPersistenceFormat.IPF_GIF, InkPersistenceCompressionMode.IPCM_Default);
using (MemoryStream ms = new MemoryStream(bytes))
{
using (Bitmap gif = new Bitmap(ms))
{
gif.Save(tempFile);
}
}
}
This saves the ink, but there is no background image.
How do I save both the ink and image into a single file?
Any help or direction pointing is appreciated...
### EDIT
Here is what else I have tried so far
InkRectangle inkBox = axInkPicture1.Ink.GetBoundingBox();
byte[] gifbits = (byte[])axInkPicture1.Ink.Save(InkPersistenceFormat.IPF_GIF, InkPersistenceCompressionMode.IPCM_Default);
using (System.IO.MemoryStream buffer = new System.IO.MemoryStream(gifbits))
using (var inkImage = new Bitmap(buffer))
using (var picture = new Bitmap(axInkPicture1.Picture))
using (var g = Graphics.FromImage(picture))
{
g.DrawImage(inkImage, new Rectangle(inkBox.Left, inkBox.Right, axInkPicture1.Picture.Width, axInkPicture1.Height));
picture.Save(tempFile, System.Drawing.Imaging.ImageFormat.Gif);
}
...but I'm still not having any luck. This saves only the background picture and not the ink. Any ideas what I'm doing wrong?
Thanks in advance

Soooo.... after much head banging and gnashing of teeth, I finally came up with a solution to my problem, although maybe not the solution I had expected and maybe not the best solution, but it works. I'm new to the world of c# and .net so please feel free to correct me me if I state anything stupid here.
What I found was that the code in my edit above was actually working, I just couldn't see the ink because it was outside the bounds of the rectangle. Apparently rectangles acquired using the GetBoundingBox() method gives you inkSpace coordinates which are vastly different than pixel coordinates.
With that in mind, my next goal became to convert the inkSpace coordinates to pixel coordinates using the inkSpaceToPixels method. I struggled with this for awhile and could never figure out how to do this using the activex version of inkPicure or axInkPicture. It required some handle to hdcDisplay which I couldn't quite wrap my head around.
In the end, I used the .net version (Microsoft.Ink) from Microsoft.Ink.dll. The only reason I was using the activeX version to begin with was because I couldn't get the Windows Tablet PC SDK to install on my development machine for some reason. It kept complaining that I needed the WinXP Tablet PC OS to install it. I figured out that I was able to install it on another (Win7) PC and then just copy the .dlls over from that PC.
So, here is the final version that is now working... whew!
//set the temporary file paths
string tempPath = Path.GetTempPath();
string tempFile = tempPath + #"tmpFile.png";
//get the bounding box and reference point for the ink strokes (in inkspace coordinates!)
Rectangle inkBox = inkPicture1.Ink.GetBoundingBox();
Point point = new Point(inkBox.Left, inkBox.Top);
MessageBox.Show(point.X.ToString() + ", " + point.Y.ToString());//"before" coordinates
// Convert inkspace coordinates to pixel coordinates using Renderer
Graphics tempGraphics = CreateGraphics();
Microsoft.Ink.Renderer inkRenderer = new Renderer();
inkRenderer.InkSpaceToPixel(tempGraphics, ref point);;
MessageBox.Show(point.X.ToString() + ", " + point.Y.ToString());//"after" coordinates
// Clean up
tempGraphics.Dispose();
// save the ink and background image to a temp file
if (inkPicture1.Ink.Strokes.Count > 0)
{
byte[] bytes = inkPicture1.Ink.Save(PersistenceFormat.Gif, CompressionMode.NoCompression);
using (MemoryStream ms = new MemoryStream(bytes))
{
using (Bitmap gif = new Bitmap(ms))
using (var bmp = new Bitmap(inkPicture1.BackgroundImage))
using (var g = Graphics.FromImage(bmp))
{
g.DrawImage(gif, new Rectangle(point.X, point.Y, gif.Width, gif.Height));
bmp.Save(tempFile, System.Drawing.Imaging.ImageFormat.Png);
}
}
}
Sorry for the long winded explanation, but I felt that simply posting the solution without adequate explanation would leave too many open questions...

I guess I don't have enough 50 reputation to comment, so I will answer with this comment for others that might land here after a google search like I did:
This was very helpful. I translated to VB and it worked.
After some flailing around I found the DLL at:
C:\Program Files (x86)\Common Files\Microsoft Shared\Ink\Microsoft.Ink.dll
Apparently I am not the only one to have a problem locating the DLL as there are many google entries on how to get it -- many are wrong and most are not this simple.
If you are just starting out with signature capture, remember to add a background image to your control or it will fail on this line:
Using bmp = New Bitmap(InkPicture1.BackgroundImage)

WinRT image handling

A friend and I spent the better part of last night nearly tearing our hair out trying to work with some images in a metro app. We got images into the app with the share charm, and then I wanted to do some other work with them, cropping the images and saving them back into the appdata folder. This proved extremely frustrating.
My question, at the end of all this, is going to be "What's the proper way of doing this, without feeling like I'm hammering together a bunch of mismatched jigsaw puzzle pieces?"
When sharing multiple images with the app, they come in as a list of Windows.Storage.StorageFiles. Here's some code used to handle that.
var storageItems = await _shareOperation.Data.GetStorageItemsAsync();
foreach (StorageFile item in storageItems)
{
var stream = await item.OpenReadAsync();
var properties = await item.Properties.GetImagePropertiesAsync();
var image = new WriteableBitmap((Int32)properties.Width, (Int32)properties.Height);
image.SetSource(stream);
images.Add(image);
}
Some searching online has indicated that currently, a Windows.UI.Xaml.Media.Imaging.WriteableBitmap is the only thing capable of letting you access the pixel data in the image. This question includes a helpful answer full of extension methods for saving images to a file, so we used those.
Our problems were the worst when I tried opening the files again later. I did something similar to before:
var files = await ApplicationData.Current.LocalFolder.GetFilesAsync();
foreach (var file in files)
{
var fileStream = await file.OpenReadAsync();
var properties = await file.Properties.GetImagePropertiesAsync();
var bitmap = new WriteableBitmap((Int32)properties.Width, (Int32)properties.Height);
bitmap.SetSource(fileStream);
System.IO.Stream stream = bitmap.PixelBuffer.AsStream();
Here comes a problem. How long is this stream, if I want the bytes out of it?
// CRASH! Length isn't supported on an IRandomAccessStream.
var pixels = new byte[fileStream.Length];
Ok try again.
var pixels = new byte[stream.Length];
This works, except... if the image is compressed, the stream is shorter than you would expect, so you will eventually get an out of bounds exception. For now pretend it's an uncompressed bitmap.
await _stream.ReadAsync(pixels, 0, pixels.Length);
Well guess what. Even though I said bitmap.SetSource(fileStream); in order to read in the data, my byte array is still full of zeroes. I have no idea why. If I pass this same bitmap into a my UI through the sample data group, the image shows up just fine. So it has clearly got the pixel data in that bitmap somewhere, but I can't read it out of bitmap.PixelBuffer? Why not?
Finally, here's what ended up actually working.
var decoder = await BitmapDecoder.CreateAsync(BitmapDecoder.PngDecoderId, fileStream);
var data = await decoder.GetPixelDataAsync();
var bytes = data.DetachPixelData();
/* process my data, finally */
} // end of that foreach I started a while ago
So now I have by image data, but I still have a big problem. In order to do anything with it, I have to make assumptions about its format. I have no idea whether it's rgba, rgb, abgr, bgra, whatever they can be. If I guess wrong my processing just fails. I've had dozens of test runs spit out zero byte and corrupted images, upside down images (???), wrong colors, etc. I would have expected to find some of this info in the properties that I got from calling await file.Properties.GetImagePropertiesAsync();, but no luck. That only contains the image width and height, plus some other useless things. Minimal documentation here.
So, why is this process so painful? Is this just reflecting the immaturity of the libraries right now, and can I expect it to get better? Or is there already some standard way of doing this? I wish it were as easy as in System.Drawing. That gave you all the data you ever needed, and happily loaded any image type correctly, without making you deal with streams yourself.

From what I have seen - when you are planning on loading the WriteableBitmap with a stream - you don't need to check the image dimensions - just do new WriteableBitmap(1,1), then call SetSource().
Not sure why you were thinking var pixels = new byte[fileStream.Length]; would work, since the fileStream has the compressed image bytes and not a pixel array.
You might need to seek to the beginning of the stream to get the pixels array:
var pixelStream = pixelBuffer.AsStream();
var bytes = new byte[this.pixelStream.Length];
this.pixelStream.Seek(0, SeekOrigin.Begin);
this.pixelStream.Read(bytes, 0, Bytes.Length);
I had started working on a WinRT port of WriteableBitmapEx - maybe it could help you: http://bit.ly/WriteableBitmapExWinRT. I have not tested it well and it is based on an older version of WBX, but it is fairly complete in terms of feature support. Might be a tad slower than it is possible too.

When drawing an image: System.Runtime.InteropServices.ExternalException: A generic error occurred in GDI

I've got a global Graphics object created from a Panel. At regular intervals an image is picked up from the disk and drawn into the panel using Graphics.DrawImage(). It works fine for a few iterations and then I'm getting the following helpful exception:
System.Runtime.InteropServices.ExternalException: A generic error occurred in GDI+.
at System.Drawing.Graphics.CheckErrorStatus(Int32 status)
at System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y)
at System.Drawing.Graphics.DrawImage(Image image, Point point)
I ruled out memory leaks as I dispose of the image object when I'm done with it. I know that the images are not corrupted and can be read fine as the program executes fine for a while before the panel stops showing.
I ran into the same problem when using a PictureBox but this time at least I got an error instead of nothing.
I checked the GDI objects and USER objects in the Task Manager but they're always around 65 user objects and 165 GDI objects when the app works and when it doesn't.
I do need to get to the bottom of this as soon as and it's not like I can stick breakpoints in .NET System libraries and see where exactly execution fails.
Thanks in advance.
EDIT: This is the display code:
private void DrawImage(Image image)
{
Point leftCorner = new Point((this.Bounds.Width / 2) - (image.Width / 2), (this.Bounds.Height / 2) - (image.Height / 2));
_graphics.DrawImage(image, leftCorner);
}
the image load code:
private void LoadImage(string filename, ref Image image)
{
MemoryStream memoryStream = DecryptImageBinary(Settings.Default.ImagePath + filename, _cryptPassword);
image = Image.FromStream(memoryStream);
memoryStream.Close();
memoryStream.Dispose();
memoryStream = null;
}
_image is global and its reference is updated in LoadImage. They are passed as parameters as I want to change the global references from as few places as possible only and keep the other methods self contained. _graphics is also global.
I've also got a webBrowser control for web sites and I either show an image or a website at one time. when there's time to display an image, the following code executes:
webBrowser.Visible = false;
panel.Visible = true;
DrawImage(_image)
_image.Dispose();
_image = null;
_image is referencing a pre-loaded image.
Hope this helps.

Your problem is similar to what I thought, but not quite. When you are loading the image, you are loading it from a MemoryStream. You have to keep the stream open for the lifetime of the image, see MSDN Image.FromStream.
You must keep the stream open for the lifetime of the Image.
The solution is to make a copy of your image in the FromImage function:
private void LoadImage(string filename, ref Image image)
{
using (MemoryStream memoryStream = DecryptImageBinary(Settings.Default.ImagePath + filename, _cryptPassword))
{
using (tmpImage = Image.FromStream(memoryStream))
{
image = new Bitmap(tmpImage);
}
}
}
Similar to the dispose problem I mentioned, the image will seem to work and then randomly fail when the underlying stream is garbage collected.

Without a little more code there is not enough to properly diagnose here, however, one thing to look at is that you may have disposed on the image your a drawing with at some point earlier and it is only after the garbage collector runs that your code is failing. Are you using cloned images anywhere? One thing I was suprised to learn is that if you do a straight clone of an image, you are not cloning the underlying bitmap that the image rely's upon, only the image structure, to create a proper copy of an image you have to create a new image:
var newImage = new Bitmap(img)
as
var newImage = oldImg.Clone();
oldImg.Dispose();
...
gr.DrawImage(newImage, new Rectangle(0,0,newImage.Width,newImage.Height);
will work for a while, but then fail at some random point...