C# and Kinect v2: Get RGB values that fit to depth-pixel - c#

I played a bit around with the Kinect v2 and C# and tried to get a 512x424 pixel-sized image array that contains depth data aswell as the regarding color information (RGBA).
Therefore I used the MultiSourceFrameReader class to receive a MultiSourceFrame from which I got the ColorFrame and DepthFrame. With the methods ColorFrame.CopyConvertedFrameDataToArray() and DepthFrame.CopyFrameDataToArray() I received the arrays that hold color and depth information:
// Contains 4*1920*1080 entries of color-info: BGRA|BGRA|BGRA..
byte[] cFrameData = new byte[4 * cWidth * cHeight];
cFrame.CopyConvertedFrameDataToArray(cFrameData, ColorImageFormat.Bgra);
// Has 512*424 entries with depth information
ushort[] dFrameData = new ushort[dWidth* dHeight];
dFrame.CopyFrameDataToArray(dFrameData);
Now I would have to map the color-quadruples that live within the ColorFrame-data-array cFrameData to each of the entries of the DepthFrame-data-array dFrameData but that's where I'm stuck. Output should be an array that is 4 times (RGBA/BGRA) the size of the dFrameData array and contains the color information to each pixel of the depth-frame:
// Create the array that contains the color information for every depth-pixel
byte[] dColors = new byte[4 * dFrameData.Length];
for (int i = 0, j = 0; i < cFrameData.Length; ++i)
{
// The mapped color index. ---> I'm stuck here:
int colIx = ?;
dColors[j] = cFrameData[colIx]; // B
dColors[j + 1] = cFrameData[colIx + 1]; // G
dColors[j + 2] = cFrameData[colIx + 2]; // R
dColors[j + 3] = cFrameData[colIx + 3]; // A
j += 4;
}
Does anyone have any suggestions?
I also took a look at the Kinect-SDK's CoordinateMappingBasics example but they did it vice versa for the 1920x1080 pixel-sized image which I already got to work.
Edit
I recognized that I should be able to get the mapped color information by using the ColorSpacePoint-struct which contains the X and Y coordinates to the specific color pixel. Therefore I set up the points like..
// Lookup table for color-point information
ColorSpacePoint[] cSpacePoints = new ColorSpacePoint[dWidth * dHeight];
this.kinectSensor.CoordinateMapper.MapDepthFrameToColorSpace(dFrameData, cSpacePoints);
.. and tried to access the color information like ..
int x = (int)(cSpacePoints[i].X + 0.5f);
int y = (int)(cSpacePoints[i].Y + 0.5f);
int ix = x * cWidth + y;
byte r = cFrameData[ix + 2];
byte g = cFrameData[ix + 1];
byte b = cFrameData[ix];
byte a = cFrameData[ix + 3];
.. but I'm still getting the wrong colors. Mostly white ones.

Well, I figured it out by myself. The error was trivial. As the array is not a pixel-array where one entry contains RGBA information but a byte array where each entry represents either R, G, B or A I had to multiply the index by the bytes-per-pixel value which in this case is 4. So the solution looks like:
int ix = (x * cWidth + y) * 4;
byte r = cFrameData[ix + 2];
byte g = cFrameData[ix + 1];
byte b = cFrameData[ix];
byte a = cFrameData[ix + 3];

Related

C# - Padding image bytes with white bytes to fill 512 x 512

I'm using Digital Persona SDK to scan fingerprints in wsq format, for requeriment I need 512 x 512 image, the SDK only export 357 x 392 image.
The sdk provide a method to compress captured image from device in wsq format and return a byte array that I can write to disk.
-I've tried to allocate a buffer of 262144 for 512 x 512 image.
-Fill the new buffer with white pixel data each byte to value 255.
-Copy the original image buffer into the new image buffer. The original image doesn’t need to be centered but it's important to make sure to copy without corrupting the image data.
To summarize I've tried to copy the old image into the upper right corner of the new image.
DPUruNet.Compression.Start();
DPUruNet.Compression.SetWsqBitrate(95, 0);
Fid capturedImage = captureResult.Data;
//Fill the new buffer with white pixel data each byte to value 255.
byte[] bytesWSQ512 = new byte[262144];
for (int i = 0; i < bytesWSQ512.Length; i++)
{
bytesWSQ512[i] = 255;
}
//Compress capturedImage and get bytes (357 x 392)
byte[] bytesWSQ = DPUruNet.Compression.CompressRaw(capturedImage.Views[0].Width, capturedImage.Views[0].Height, 500, 8, capturedImage.Views[0].RawImage, CompressionAlgorithm.COMPRESSION_WSQ_NIST);
//Copy the original image buffer into the new image buffer
for (int i = 0; i < capturedImage.Views[0].Height; i++)
{
for (int j = 0; j < capturedImage.Views[0].Width; j++)
{
bytesWSQ512[i * bytesWSQ512.Length + j ] = bytesWSQ[i * capturedImage.Views[0].Width + j];
}
}
//Write bytes to disk
File.WriteAllBytes(#"C:\Users\Admin\Desktop\bytesWSQ512.wsq", bytesWSQ512);
DPUruNet.Compression.Finish();
When running that snippet I get IndexOutOfRangeException, I don't know if the loop or the calculation of indexes for new array are right.
Here is a representation of what I'm trying to do.
If someone is trying to achieve something like this or padding a raw image, I hope this will help.
DPUruNet.Compression.
DPUruNet.Compression.SetWsqBitrate(75, 0);
Fid ISOFid = captureResult.Data;
byte[] paddedImage = PadImage8BPP(captureResult.Data.Views[0].RawImage, captureResult.Data.Views[0].Width, captureResult.Data.Views[0].Height, 512, 512, 255);
byte[] bytesWSQ512 = Compression.CompressRaw(512, 512, 500, 8, paddedImage, CompressionAlgorithm.COMPRESSION_WSQ_NIST);
And the method to resize (pad) the image is:
public byte[] PadImage8BPP(byte[] original, int original_width, int original_height, int desired_width, int desired_height, byte pad_color)
{
byte[] canvas_8bpp = new byte[desired_width * desired_height];
for (int i = 0; i < canvas_8bpp.Length; i++)
canvas_8bpp[i] = pad_color; //Fill background. Note this type of fill will fail histogram checks.
int clamp_y_begin = 0;
int clamp_y_end = original_height;
int clamp_x_begin = 0;
int clamp_x_end = original_width;
int pad_y = 0;
int pad_x = 0;
if (original_height > desired_height)
{
int crop_distance = (int)Math.Ceiling((original_height - desired_height) / 2.0);
clamp_y_begin = crop_distance;
clamp_y_end = original_height - crop_distance;
}
else
{
pad_y = (desired_height - original_height) / 2;
}
if (original_width > desired_width)
{
int crop_distance = (int)Math.Ceiling((original_width - desired_width) / 2.0);
clamp_x_begin = crop_distance;
clamp_x_end = original_width - crop_distance;
}
else
{
pad_x = (desired_width - original_width) / 2;
}
//We traverse the captured image (either whole image or subset)
for (int y = clamp_y_begin; y < clamp_y_end; y++)
{
for (int x = clamp_x_begin; x < clamp_x_end; x++)
{
byte image_pixel = original[y * original_width + x];
canvas_8bpp[(pad_y + y - clamp_y_begin) * desired_width + pad_x + x - clamp_x_begin] = image_pixel;
}
}
return canvas_8bpp;
}

How to crop image that is a byte array

I have an Azure Function and would want to crop an image that is inside a byte array, and then save it to a blob. I am not sure how to do the cropping part, because Bitmap is not available in Azure Functions. Here is my saving to blob:
private async static Task<string> CreateBlob(string name, byte[] data, TraceWriter log)
which works fine for what it does. Also, I have the URL of the image I want to crop, if that helps. Just don't know how to achieve this with an Azure Function. Any ideas?
EDIT
this is the original image, that I have inside a byte array. I want to crop another one from it with the size - 650x290
int cropStartX = 0;
int cropStartY = 0;
int cropWidth = 650;
int cropHeight = 290;
int multiply = cropWidth * cropHeight;
byte[] croppedImage = new byte[multiply];
for (int j = cropStartX; j < cropStartX + cropWidth; j++)
{
for (int k = cropStartY; k < cropStartY + cropHeight; k++)
{
if ((j + k * cropWidth) >= ((byte[])OriginalPic).Length)
break;
croppedImage[j + k * cropWidth] = ((byte[])OriginalPic)[j + k * cropWidth];
}
}
This is the size of the OriginalPic - byte[49121]
This is the output if I set:
croppedImage[j + k * cropWidth] = ((byte[])OriginalPic)[j + k * (cropWidth + 20)];
idk if this is what you are after, but assuming you know the dimensions of your picture, you just have to encode that into your index :
pseudocode
int cropWidth = 100;
int cropHeight = 50;
byte[cropWidth*cropHeight] croppedImage;
for(int i=cropStartX;i<cropStartX+cropWidth;i++)
{
for(int ii = cropStartY;ii<cropStartY+cropHeight;ii++)
{
croppedImage[i + ii*cropWidth] = image[i + ii*mainImageWidth];
}
}
I know this isnt as fast as if you can actually just do a memory copy function... but will do what you need.
Update: after looking at more of your information I am afraid that this method will not work for you as you have compressed image data. Basically jpeg compresses data at a size of about 1:10 which makes more sense now that your image byte array is only 49k as that is almost exactly the file size, which contains jpeg compressed information for a 1024x434 picture of rgb. Long story short you can not simply query pixel locations from that array to get meaningful data unless you were to uncompressed the image prior to cropping. Apologies as I should have seen this as soon as I noticed a "byte" was supposedly representing an rgb pixel (which does not match).

Converting CDF of histogram to c# from matlab?

How can I convert this matlab code to AForge.net+c# code?
cdf1 = cumsum(hist1) / numel(aa);
I found that there is Histogram.cumulative method is present in Accord.net.
But I dont know how to use.
Please teaching how to convert.
% Histogram Matching
%
clear
clc
close all
pkg load image
% 이미지 로딩
aa=imread('2.bmp');
ref=imread('ref2.png');
figure(1); imshow(aa); colormap(gray)
figure(2); imshow(ref); colormap(gray)
M = zeros(256,1,'uint8'); % Store mapping - Cast to uint8 to respect data type
hist1 = imhist(aa); % Compute histograms
hist2 = imhist(ref);
cdf1 = cumsum(hist1) / numel(aa); % Compute CDFs
cdf2 = cumsum(hist2) / numel(ref);
% Compute the mapping
for idx = 1 : 256
[~,ind] = min(abs(cdf1(idx) - cdf2));
M(idx) = ind-1;
end
% Now apply the mapping to get first image to make
% the image look like the distribution of the second image
out = M(double(aa)+1);
figure(3); imshow(out); colormap(gray)
Actually, I don't have a great knowledge of Accord.NET, but reading the documentation I think that ImageStatistics class is what you are looking for (reference here). The problem is that it cannot build a single histogram for the image and you have to do it by yourself. imhist in Matlab just merges the three channels and then counts the overall pixel occurrences so this is what you should do:
Bitmap image = new Bitmap(#"C:\Path\To\Image.bmp");
ImageStatistics statistics = new ImageStatistics(image);
Double imagePixels = (Double)statistics.PixelsCount;
Int32[] histR = statistics.Red.Values.ToArray();
Int32[] histG = statistics.Green.Values.ToArray();
Int32[] histB = statistics.Blue.Values.ToArray();
Int32[] histImage = new Int32[256];
for (Int32 i = 0; i < 256; ++i)
histImage[i] = histR[i] + histG[i] + histB[i];
Double cdf = new Double[256];
cdf[0] = (Double)histImage[0];
for (Int32 i = 1; i < 256; ++i)
cdf[i] = (Double)(cdf[i] + cdf[i - 1]);
for (Int32 i = 0; i < 256; ++i)
cdf[i] = cdf[i] / imagePixels;
In C#, an RGB value can be built from R, G and B channel values as follows:
public static int ChannelsToRGB(Int32 red, Int32 green, Int32 blue)
{
return ((red << 0) | (green << 8) | (blue << 16));
}

Microsoft OCR: Convert bitmap to pixel array for Windows Phone 8 Silverlight

I have been searching for answer for 2 days but i couldn't. That's why i'm posting it here.
I followed this tutorial.
I got error bitmap.SetSource(imgStream); so i changed it to bitmap.SetSource(imgStream.AsStream);
I also got error message on this line. I'm unable to convert pixel to array. Because there is no PixelBuffer and i cannot use Pixels
var ocrResult = await ocrEngine.RecognizeAsync((uint)bitmap.PixelHeight, (uint)bitmap.PixelWidth, bitmap.PixelBuffer.ToArray());
So i searched on internet and found this link of stackoverflow.com. So I copied and pasted the following code
public static byte[] ToByteArray(this WriteableBitmap bmp)
{
// Init buffer
int w = bmp.PixelWidth;
int h = bmp.PixelHeight;
int[] p = bmp.Pixels;
int len = p.Length;
byte[] result = new byte[4 * w * h];
// Copy pixels to buffer
for (int i = 0, j = 0; i < len; i++, j += 4)
{
int color = p[i];
result[j + 0] = (byte)(color >> 24); // A
result[j + 1] = (byte)(color >> 16); // R
result[j + 2] = (byte)(color >> 8); // G
result[j + 3] = (byte)(color); // B
}
return result;
}
And then byte[] hello = ByteArrayChange.ToByteArray(bitmap);
var ocrResult = await ocrEngine.RecognizeAsync((uint)bitmap.PixelHeight, (uint)bitmap.PixelWidth, hello );
I run the code with Device and it gave the exception in App.Xaml.cs in Application_UnhandledException
Note: I'm developing on Windows Phone 8/8.1 (Silverlight)
OCR documentation on MSDN contains part about Silverlight apps.

Get all pixel information of an image efficiently

I made a program to get all image pixel RGB color codes from picture. Basically, it sets y position on constant and changes x position zero to width and also y by looping.
Ok it's work, but the problem is it take more than 20 minutes to get all pixel from even (1000*604 height width) image. Please anyone help?
I'm surprised if this process takes so long, then how can we make a program like bar-code reader from image. I want to get all pixel value from image, here is my C# code are below.
I also uploaded my program here, check it if you don't agree.
void myimage() {
mypic = new Bitmap(pathname);
int imwid = mypic.Width;
int imhei = mypic.Height;
int total=imwid*imhei;
for (int z = 0; z <imhei;z++ )
{
for (int i = 0; i < imwid; i++)
{
Color pixelColor = mypic.GetPixel(i, z);
textBox2.AppendText(" " + pixelColor.R +
" " + pixelColor.G +
" " + pixelColor.B + " " +
pixelColor.A +
Environment.NewLine);
}
}
}
Take a look at this:
var data = mypic.LockBits(
new Rectangle(Point.Empty, mypic.Size),
ImageLockMode.ReadWrite, mypic.PixelFormat);
var pixelSize = data.PixelFormat == PixelFormat.Format32bppArgb ? 4 : 3; // only works with 32 or 24 pixel-size bitmap!
var padding = data.Stride - (data.Width * pixelSize);
var bytes = new byte[data.Height * data.Stride];
// copy the bytes from bitmap to array
Marshal.Copy(data.Scan0, bytes, 0, bytes.Length);
var index = 0;
var builder = new StringBuilder();
for (var y = 0; y < data.Height; y++)
{
for (var x = 0; x < data.Width; x++)
{
Color pixelColor = Color.FromArgb(
pixelSize == 3 ? 255 : bytes[index + 3], // A component if present
bytes[index + 2], // R component
bytes[index + 1], // G component
bytes[index] // B component
);
builder
.Append(" ")
.Append(pixelColor.R)
.Append(" ")
.Append(pixelColor.G)
.Append(" ")
.Append(pixelColor.B)
.Append(" ")
.Append(pixelColor.A)
.AppendLine();
index += pixelSize;
}
index += padding;
}
// copy back the bytes from array to the bitmap
Marshal.Copy(bytes, 0, data.Scan0, bytes.Length);
textBox2.Text = builder.ToString();
Is just an example, read some good tutorials about LockBits and imaging in general to understand clearly what happens.
Getting pixel information shouldn't take that long. Can you log the time it takes myimage() to run? The slowness might be somewhere else. Also try removing the line that begins with textBox2.AppendText in myimage() and see how fast it runs then.

Categories