We Keep Coding

Enhance performance to paint image, is SIMD perhapse a solution?

I have no experience with SIMD, but have a method that is too slow. I know get 40fps, and I need more.
Does anyone know how I could make this paint method faster? Perhaps the SIMD instructions are a solution?
The sourceData is now a byte[] (videoBytes) but could use a pointer too.
public bool PaintFrame(IntPtr layerBuffer, ushort vStart, byte vScale)
{
for (ushort y = 0; y < height; y++)
{
ushort eff_y = (ushort)(vScale * (y - vStart) / 128);
var newY = tileHeight > 0 ? eff_y % tileHeight : 0;
uint y_add = (uint)(newY * tileWidth * bitsPerPixel >> 3);
for (int x = 0; x < width; x++)
{
var newX = tileWidth > 0 ? x % tileWidth : 0;
ushort x_add = (ushort)(newX * bitsPerPixel >> 3);
uint tile_offset = y_add + x_add;
byte color = videoBytes[tile_offset];
var colorIndex = BitsPerPxlCalculation(color, newX);
// Apply Palette Offset
if (paletteOffset > 0)
colorIndex += paletteOffset;
var place = x + eff_y * width;
Marshal.WriteByte(layerBuffer + place, colorIndex);
}
}
return true;
}
private void UpdateBitPerPixelMethod()
{
// Convert tile byte to indexed color
switch (bitsPerPixel)
{
case 1:
BitsPerPxlCalculation = (color, newX) => color;
break;
case 2:
BitsPerPxlCalculation = (color, newX) => (byte)(color >> 6 - ((newX & 3) << 1) & 3);
break;
case 4:
BitsPerPxlCalculation = (color, newX) => (byte)(color >> 4 - ((newX & 1) << 2) & 0xf);
break;
case 8:
BitsPerPxlCalculation = (color, newX) => color;
break;
}
}
More info
Depending on the settings, the bpp can be changed. The indexed colors and the palette colors are separatly stored. Here I have to recreate the image pixels indexes, so later on I use the palette and color indexes in WPF(Windows) or SDL(Linux, Mac) to display the image.
vStart is the ability to crop the image on top.
The UpdateBitPerPixelMethod() will not change during a frame rendering, only before. During the for, no settings data can be changed.
So I was hoping that some parts can be written with SIMD, because the procedure is the same for all pixels.

Hy,
your code is not the clearest to me. Are you trying to create a new matrix / image ? If yes create a new 2D allocation and calculate the entire image into it. Set it to 0 after you do not need the calculations anymore.
Replace the Marshal.WriteByte(layerBuffer + place, colorIndex);with a 2D image ( maybe this is the image ?).
Regarding the rest it is a problem because you have non uniform offsets in indexing and jumps. That will make developing a SIMD solution difficult (you need masking and stuff). My bet would be to calculate everything for all the indices and save it into individual 2D matrices, that are allocated once at the begining.
For example:
ushort eff_y = (ushort)(vScale * (y - vStart) / 128);
Is calculated per every image row. Now you could calculate it once as an array since I do not believe that the format size of the images changes during the run.
I dont know if vStart and vScale are defined as a constant at program start. You should do this for every calculation that uses constant, and just read the matrices later to calculate.
SIMD can help but only if you do every iteration you calculate the same thing and if you avoid branching and switch cases.
Addition 1
You have multiple problems and design considerations from my stand point.
First of all you need to get away from the idea SIMD is going to help in your case. You would need to remove all conditional statements. SIMD-s are not build to deal with conditional statements.
Your idea should be to split up the logic into manageable pieces so you can see witch piece of the code takes most time.
One big problem is the write byte in the marshal, this is automatically saying to the compiler that you handle only and exclusively 1 byte. I'm guessing that this creates on big bottle neck.
By code analysis I see in each loop you are doing checks. This must be restructured.
Assumption is the image get rarely cropped this would be a separation from the image calculations.
List<ushort> eff_y = new List<ushort>();
List<uint> y_add = new List<uint>();
for (ushort y = 0; y < height; y++)
{
eff_y.add((ushort)(vScale * (y - vStart) / 128));
var newY = tileHeight > 0 ? eff_y % tileHeight : 0;
y_add = (uint)(newY * tileWidth * bitsPerPixel >> 3);
}
So this can be precalculated and changed only when the cropping changes.
Now it gets realy tricky.
paletteOffset - the if statement makes only sense in paletteOffset can be negative, then zero it out and remove the if statement
bitsPerPixel - this looks like a fixed value for the rendering duration
so remove the UpdateBitPerPixelMethod and send in a parameter.
for (ushort y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
var newX = tileWidth > 0 ? x % tileWidth : 0; // conditional stetement
ushort x_add = (ushort)(newX * bitsPerPixel >> 3);
uint tile_offset = y_add + x_add;
byte color = videoBytes[tile_offset];
var colorIndex = BitsPerPxlCalculation(color, newX);
// Apply Palette Offset
if (paletteOffset > 0) // conditional stetement
colorIndex += paletteOffset;
var place = x + eff_y * width;
Marshal.WriteByte(layerBuffer + place, colorIndex);
}
}
This are only few things that need to be done before you try anything with the SIMD. But by that time the changes will give the compiler hints about what you want to do. This could improve the machine code execution. You need also to test the performance of your code to pinpoint the bottle neck it is very hard to assume or guess correctly by code.
Good luck

Turning int array into BMP in C#

I'm having problems converting a grayscale array of ints (int32[,]) into BMP format in C#.
I tried cycling through the array to set pixel color in the BMP, it does work but it ends up being really slow and practically unusable.
I did a lot of googling but I cannot find the answer to my question.
I need to put that image in a PictureBox in real time so the method needs to be fast.
Relevant discussion here
Edit: the array is 8bit depth but stored as int32
Edit2: Just found this code
private unsafe Task<Bitmap> BitmapFromArray(Int32[,] pixels, int width, int height)
{
return Task.Run(() =>
{
Bitmap bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb);
BitmapData bitmapData = bitmap.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
for (int y = 0; y < height; y++)
{
byte* row = (byte*)bitmapData.Scan0 + bitmapData.Stride * y;
for (int x = 0; x < width; x++)
{
byte grayShade8bit = (byte)(pixels[x, y] >> 4);
row[x * 3 + 0] = grayShade8bit;
row[x * 3 + 1] = grayShade8bit;
row[x * 3 + 2] = grayShade8bit;
}
}
bitmap.UnlockBits(bitmapData);
return bitmap;
});
}
Seems to work fast enough but the image is almost black. If I remove the top of the camera the Image should be completely white but it just displays a really dark grey. I guess it's interpreting the pixel value as 32bit, not 8bit. Then tried to cast (ushort)pixels[x, y] but doesn't work

I actually wrote a universally usable BuildImagefunction here on SO to build an image out of a byte array, but of course, you're not starting from a byte array, you're starting from a two-dimensional Int32 array. The easy way to get around it is simply to transform it in advance.
Your array of bytes-as-integers is a rather odd thing. If this is read from a grayscale image I'd rather assume this is 32-bit ARGB data, and you're just using the lowest component of each value (which would be the blue one), but if downshifting the values by 4 bits produced uniformally dark values I'm inclined to take your word for that; otherwise the bits of the next colour component (green) would bleed in, giving bright colours as well.
Anyway, musing and second-guessing aside, here's my actual answer.
You may think each of your values, when poured into an 8-bit image, is simply the brightness, but this is actually false. There is no specific type in the System.Drawing pixel formats to indicate 8-bit grayscale, and 8-bit images are paletted, which means that each value on the image refers to a colour on the colour palette. So, to actually make an 8-bit grayscale image where your byte values indicate the pixel's brightness, you'll need to explicitly define a colour palette where the indices of 0 to 255 on the palette contain gray colours going from (0,0,0) to (255,255,255). Of course, this is pretty easy to generate.
This code will transform your array into an 8-bit image. It uses the aforementioned BuildImage function. Note that that function uses no unsafe code. The use of Marshal.Copy means raw pointers are never handled directly, making the code completely managed.
public static Bitmap FromTwoDimIntArrayGray(Int32[,] data)
{
// Transform 2-dimensional Int32 array to 1-byte-per-pixel byte array
Int32 width = data.GetLength(0);
Int32 height = data.GetLength(1);
Int32 byteIndex = 0;
Byte[] dataBytes = new Byte[height * width];
for (Int32 y = 0; y < height; y++)
{
for (Int32 x = 0; x < width; x++)
{
// logical AND to be 100% sure the int32 value fits inside
// the byte even if it contains more data (like, full ARGB).
dataBytes[byteIndex] = (Byte)(((UInt32)data[x, y]) & 0xFF);
// More efficient than multiplying
byteIndex++;
}
}
// generate palette
Color[] palette = new Color[256];
for (Int32 b = 0; i < 256; b++)
palette[b] = Color.FromArgb(b, b, b);
// Build image
return BuildImage(dataBytes, width, height, width, PixelFormat.Format8bppIndexed, palette, null);
}
Note, even if the integers were full ARGB values, the above code would still work exactly the same; if you only use the lowest of the four bytes inside the integer, as I said, that'll simply be the blue component of the full ARGB integer. If the image is grayscale, all three colour components should be identical, so you'll still get the same result.
Assuming you ever find yourself with the same kind of byte array where the integers actually do contain full 32bpp ARGB data, you'd have to shift out all four byte values, and there would be no generated gray palette, but besides that, the code would be pretty similar. Just, handling 4 bytes per X iteration.
public static Bitmap fromTwoDimIntArrayGray(Int32[,] data)
{
Int32 width = data.GetLength(0);
Int32 height = data.GetLength(1);
Int32 stride = width * 4;
Int32 byteIndex = 0;
Byte[] dataBytes = new Byte[height * stride];
for (Int32 y = 0; y < height; y++)
{
for (Int32 x = 0; x < width; x++)
{
// UInt32 0xAARRGGBB = Byte[] { BB, GG, RR, AA }
UInt32 val = (UInt32)data[x, y];
// This code clears out everything but a specific part of the value
// and then shifts the remaining piece down to the lowest byte
dataBytes[byteIndex + 0] = (Byte)(val & 0x000000FF); // B
dataBytes[byteIndex + 1] = (Byte)((val & 0x0000FF00) >> 08); // G
dataBytes[byteIndex + 2] = (Byte)((val & 0x00FF0000) >> 16); // R
dataBytes[byteIndex + 3] = (Byte)((val & 0xFF000000) >> 24); // A
// More efficient than multiplying
byteIndex+=4;
}
}
return BuildImage(dataBytes, width, height, stride, PixelFormat.Format32bppArgb, null, null);
}
Of course, if you want this without transparency, you can either go with three bytes as you did, or simply change PixelFormat.Format32bppArgb in the final call to PixelFormat.Format32bppRgb, which changes the meaning of the fourth byte from alpha to mere padding.

Solved (had to remove the four bits shift):
private unsafe Task<Bitmap> BitmapFromArray(Int32[,] pixels, int width, int height)
{
return Task.Run(() =>
{
Bitmap bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb);
BitmapData bitmapData = bitmap.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
for (int y = 0; y < height; y++)
{
byte* row = (byte*)bitmapData.Scan0 + bitmapData.Stride * y;
for (int x = 0; x < width; x++)
{
byte grayShade8bit = (byte)(pixels[x, y]);
row[x * 3 + 0] = grayShade8bit;
row[x * 3 + 1] = grayShade8bit;
row[x * 3 + 2] = grayShade8bit;
}
}
bitmap.UnlockBits(bitmapData);
return bitmap;
});
}
Still not sure why substituting Format24bppRgb with Format8bppIndexed doesn't work. Any clue?

Rotate raw pixel data of an image 180 degrees

I'm trying to rotate raw pixel data from a DICOM file by 180 degrees (or flipped). I've successfully flipped the image correctly, however, upon writing the pixel data back to the file (in this case it's a DICOM file) and displaying it. The final output of the image is not correct.
Below is the sample a sample of the image I'm trying to flip 180 /mirror.
Here's the code I'm using to perform the flipping:
string file = #"adicomfile.dcm";
DicomFile df = new DicomFile();
df.Load(file);
// Get the amount of bits per pixel from the DICOM header.
int bitsPerPixel = df.DataSet[DicomTags.BitsAllocated].GetInt32(0, 0);
// Get the raw pixel data from the DICOM file.
byte[] bytes = df.DataSet[DicomTags.PixelData].Values as byte[];
// Get the width and height of the image.
int width = df.DataSet[DicomTags.Columns].GetInt32(0, 0);
int height = df.DataSet[DicomTags.Rows].GetInt32(0, 0);
byte[] original = bytes;
byte[] mirroredPixels = new byte[width * height * (bitsPerPixel / 8)];
width *= (bitsPerPixel / 8);
// The mirroring / image flipping.
for (int i = 0; i < original.Length; i++)
{
int mod = i % width;
int x = ((width - mod - 1) + i) - mod;
mirroredPixels[i] = original[x];
}
df.DataSet[DicomTags.PixelData].Values = mirroredPixels;
df.Save(#"flippedicom.dcm", DicomWriteOptions.Default);
And here's my output (incorrect). The white and distortion is not the desired output.
I'm using ClearCanvas DICOM library, however this shouldn't matter as I'm only trying to manipulate the raw pixel data contained within the file itself.
The desired output would preferably look like the original, but flipped 180 / mirrored.
Some assistance would be greatly appreciated. I've tried my best searching SO, but to no avail.

It took a while, but I ended up solving my problem by using a method from a Java library. You can see the class here.
string file = #"adicomfile.dcm";
DicomFile df = new DicomFile();
df.Load(file);
// Get the amount of bits per pixel from the DICOM header.
int bitsPerPixel = df.DataSet[DicomTags.BitsAllocated].GetInt32(0, 0);
// Get the raw pixel data from the DICOM file.
byte[] bytes = df.DataSet[DicomTags.PixelData].Values as byte[];
// Get the width and height of the image.
int width = df.DataSet[DicomTags.Columns].GetInt32(0, 0);
int height = df.DataSet[DicomTags.Rows].GetInt32(0, 0);
byte[] newBytes = new byte[height * width * (bitsPerPixel / 8)];
int stride = bitsPerPixel / 8;
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width * stride; x++)
{
newBytes[((height - y - 1) * (width * stride)) + x] = bytes[(y * (width * stride)) + x];
}
}
// Set patient orientation.
df.DataSet[DicomTags.PatientOrientation].Values = #"A\L";
// The pixel data of the DICOM file to the flipped/mirrored data.
df.DataSet[DicomTags.PixelData].Values = mirroredPixels;
// Save the DICOM file.
df.Save(#"flippedicom.dcm", DicomWriteOptions.Default);
The output was correct and I was able to continue other modifications to the raw pixel data.
Thank you all for the pointers.

Reading a depth png image from a dataset

We are trying to use this American Sign Language dataset. This dataset has pictures of American Sign Language letters, both RGB and the Depth images.
I downloaded the dataset from the link. The RGB images seems fine, but the depth images are fully solid black. Something is wrong.
Since all the dataset is big, and it takes time to download all of them; I'm uploading an example RGB image and an example depth image here:
Since the depth images should have the depth data, I expect it to have float values (They say they used Kinect and Kinect provides float values). How can I read these float pixels using C#? I tried the following:
Bitmap bmp = new Bitmap("depth_0_0002.png");
int R = bmp.GetPixel(0,0).R;
int G = bmp.GetPixel(0,0).G;
int B = bmp.GetPixel(0,0).B;
However, I need float pixels, these are integer and they have nonsense values.
Do I need to include a 3rd party library?

I've tried it myself. Normally the depth datas are 16bit values.
The 13 high-order bits contain the distance and the 3 low-order bits contain the user segmentation map.
The user segmentation map is only built if skeleton tracking is active, which I believe was not in your example. Although the rgb values are 24bit it seems to work. I get an image from the segmented hand.
Bitmap bmpOrg = new Bitmap("bKawM.png");
Bitmap bmp = new Bitmap(106, 119);
for (int i = 0; i < 106;i++ )
{
for (int j = 0; j < 119;j++ )
{
Color rgb = bmpOrg.GetPixel(i, j);
int bit24 = (rgb.B << 16 + rgb.G << 8 + rgb.R);
int user = bit24 & 0x07;
int realDepth = bit24 >> 3;
bmp.SetPixel(i, j, Color.FromArgb(realDepth));
}
}
pictureBox1.Image = bmp;
My output:
I've played with it again. First I increased the brightness and contrast in Photoshop.
So the rgb values are usable if you don't need the real depth values in millimeters.
Then I tried to get the 16bit values from image with WPF because the image is 16bit grayscale encoded.
Stream imageStreamSource = new FileStream("bKawM.png", FileMode.Open, FileAccess.Read, FileShare.Read);
PngBitmapDecoder decoder = new PngBitmapDecoder(imageStreamSource, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.Default);
BitmapSource bitmapSource = decoder.Frames[0];
int height = bitmapSource.PixelHeight;
int width = bitmapSource.PixelWidth;
int stride = width * ((bitmapSource.Format.BitsPerPixel + 7) / 8);
byte[] bytes = new byte[height * stride];
bitmapSource.CopyPixels(bytes, stride, 0);
for (int x = 0; x < width; x++)
{
for (int y = 0; y < height; y++)
{
byte low = bytes[y * stride + x + 0];
byte high = bytes[y * stride + x + 1];
ushort bit16 = (ushort)((high << 8) | low);
int user = bit16 & 0x07;
int realDepth = bit16 >> 3;
}
}
I created a new image with the depth values and it looked very strange. I don't find any information
what data the image contains. I don't know if it contains the userdata (3 bits) or if the depth is converted somehow before saving to file.

Converting an array of Pixels to an image in C#

I have an array of int pixels in my C# program and I want to convert it into an image. The problem is I am converting Java source code for a program into equivalent C# code. In java the line reads which displays the array of int pixels into image:
Image output = createImage(new MemoryImageSource(width, height, orig, 0, width));
can someone tell me the C# equivalent?
Here orig is the array of int pixels. I searched the Bitmap class and there is a method called SetPixel but the problem is it takes a x,y coordinate number. But what I have in my code is an array of int pixels. Another weird thing is my orig array has negative number and they are way far away from 255. In Java this is the same case (meaning both the array in C# and Java have equivalent value) and the values is working fine in Java.
But I can't get that line translated into C#. Please help.

Using WPF, you can create a bitmap (image) directly from your array. You can then encode this image or display it or play with it:
int width = 200;
int height = 200;
//
// Here is the pixel format of your data, set it to the proper value for your data
//
PixelFormat pf = PixelFormats.Bgr32;
int rawStride = (width * pf.BitsPerPixel + 7) / 8;
//
// Here is your raw data
//
int[] rawImage = new int[rawStride * height / 4];
//
// Create the BitmapSource
//
BitmapSource bitmap = BitmapSource.Create(
width, height,
96, 96, pf, null,
rawImage, rawStride);

You can use Bitmap.LockBits to obtain the bitmap data that you can then manipulate directly, rather than via SetPixel. (How to use LockBits)

I like the WPF option already presented, but here it is using LockBits and Bitmap:
// get the raw image data
int width, height;
int[] data = GetData(out width, out height);
// create a bitmap and manipulate it
Bitmap bmp = new Bitmap(width,height, PixelFormat.Format32bppArgb);
BitmapData bits = bmp.LockBits(new Rectangle(0, 0, width, height),
ImageLockMode.ReadWrite, bmp.PixelFormat);
unsafe
{
for (int y = 0; y < height; y++)
{
int* row = (int*)((byte*)bits.Scan0 + (y * bits.Stride));
for (int x = 0; x < width; x++)
{
row[x] = data[y * width + x];
}
}
}
bmp.UnlockBits(bits);
With (as test data):
public static int[] GetData(out int width, out int height)
{
// diagonal gradient over a rectangle
width = 127;
height = 128;
int[] data = new int[width * height];
for (int x = 0; x < width; x++)
{
for (int y = 0; y < height; y++)
{
int val = x + y;
data[y * width + x] = 0xFF << 24 | (val << 16) | (val << 8) | val;
}
}
return data;
}

Well, I'm assuming each int is the composite ARGB value? If there isn't an easy option, then LockBits might be worth looking at - it'll be a lot quicker than SetPixel, but is more complex. You'll also have to make sure you know how the int is composed (ARGB? RGBA?). I'll try to see if there is a more obvious option...

MemoryImageSource's constructor's 3rd argument is an array of ints composed of argb values in that order
The example in that page creates such an array by;
pix[index++] = (255 << 24) | (red << 16) | blue;
You need to decompose that integer array to a byte array (shift operator would be useful), but it should be in bgr order, for LockBits method to work.

I would recommend using LockBits but a slower SetPixel based algorithm might look something like
// width - how many int's per row
// array - array of integers
Bitmap createImage(int width, int[] array)
{
int height = array.Length / width;
Bitmap bmp = new Bitmap(width, height);
for (int y = 0; y < height; y++)
{
for (int x = 0; x < array.Length; x += width)
{
bmp.SetPixel(x, y, Color.FromArgb(array[i]));
}
}
return bmp;
}