Writing OpenCV CV_8UC3 image matrix into writeable bitmap - c#

I have two monochromatic images as byte[] taken from cameras. I want to combine these images and write the combined image into a writeable bitmap.
Merging images: OpenCV
Using openCV, I create Mat objects (of type CV_8UC1) from the byte arrays (greenMat for the green and redMat for the red color channel image) and merge them via Cv2.Merge
Mat mergedMat = new Mat(greenMat.Height, greenMat.Width, MatType.CV_8UC3);
Mat emptyMat = Mat.Zeros(greenMat.Height, greenMat.Width, MatType.CV_8UC1);
Cv2.Merge(new[]
{
redMat, greenMat, emptyMat
}, mergedMat);
The mergedMat.ToBytes() now retuns a buffer of size 2175338. Why is it this size? Merging three one-channel matrices (CV_8UC1) if size 2448x2048 into one three-channel matrix(CV_8UC3) should yield to a buffer of size 15040512, or am I missing something here?
Display merged image: WriteableBitmap
I want to display the merged image by writing it into an existing WriteableBitmap, that is initialized via
ImageSource = new WriteableBitmap(
width, //of the green image
height, //of the green image
96, //dpiX: does not affect image as far as I tested
96, //dpiY
PixelFormats.Rgb24, //since the combined image is a three channel color image
null); //no bitmap palette needed for used PixelFormat
Is this initialization correct? I'm unsure about the PixelFormat here. The dpiX and dpiY values also seem to have no effect. What is their use?
The tricky part I can't get to work properly now is to write the image data into the WriteableBitmap.
Using the following code to obtain the byte array of the
merged image and using WriteableBitmaps' WritePixels method, with
stride according to the ImageSources' PixelFormat, yields an
System.ArgumentException: 'Buffer size is not sufficient.'
var mergedBuffer = mergedMat.ToBytes();
var bytesPerPixel = (PixelFormats.Rgb24.BitsPerPixel + 7) / 8;
var stride = bytesPerPixel * width;
((WriteableBitmap)ImageSource).WritePixels(
new Int32Rect(0, 0, width, height),
mergedBuffer,
stride,
0)
Edit: Stride is calculated according to this answer, although it's not neccessary with a fixed PixelFormat like here.
Why is the buffer too small? I did initialize the WriteableBitmap
with the correct width and height and PixelFormats.Rgb24. Am I
calculating my stride correctly?
Solution
Thanks to #Micka s suggestions in the comments I realized the mergedMat.ToBytes method seems not to do what I expceted. Instead, I tried to use mergedMat.Data pointer like so:
var mergedBufferSize = (int)(mergedMat.Total() * mergedMat.Channels());
byte[] mergedBuffer = new byte[mergedBufferSize];
System.Runtime.InteropServices.Marshal.Copy(
mergedMat.Data,
mergedBuffer,
0,
mergedBufferSize);
Everything works fine this way.

Related

C#: How to load a RAW image (format: rgb565) into a Bitmap?

My Goal:
Display an image, which comes along in rgb565 raw format, in my Windows Forms program. (off topic: the data stems from an OV7670 camera module)
My approach:
First, I create an empty Bitmap. Next, I insert the image data (raw format: rgb565) into the payload section of the empty Bitmap. Finally, I display the modified Bitmap within a PictureBox.
My problem:
Everything works fine, but the testimage is displayed with diagonal stripes instead of vertical stripes (see links below).
Original rgb565 raw image: Original rgb565 raw image
Screenshot of PictureBox (with diagonal stripes): Screenshot of PictureBox
I did manage to display the image by extracting R,G,B and using SetPixel(), but that's too slow for me. That's why I would like to get the code underneath to display the image in the correct way.
My Testimage can be found on dropbox here:
Testimage: Testimage
MemoryStream memoryStream = new MemoryStream(1000000);
// Read raw image into byte array
string imgpath = "rgb565_LSB-first_313x240.raw";
FileStream fs = new FileStream(imgpath, FileMode.Open);
fs.CopyTo(memoryStream);
Byte[] buffer = memoryStream.ToArray();
// Create empty Bitmep and inject byte arrays data into bitmap's data area
Bitmap bmp = new Bitmap(313, 240, PixelFormat.Format16bppRgb565);
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, 313, 240);
BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadWrite,
PixelFormat.Format16bppRgb565);
IntPtr ptrToFirstPixel = bmpData.Scan0;
// Inject the rgb565 data (stored in the buffer array)
Marshal.Copy(buffer, 0, ptrToFirstPixel, buffer.Length);
bmp.UnlockBits(bmpData);
// Diplay Bitmap in my PictureBox
pbImage.Image = bmp;
Expected result: vertical stripes :-)
Actual result: diagonal stripes :-(
After 10 hours poking around in the haystack, I could finally track down
the reason, which was definitly not a banality (at least not to me).
Here it comes:
Bitmap specification requires, to pad row size to a multiple of 4 Bytes!
https://upload.wikimedia.org/wikipedia/commons/c/c4/BMPfileFormat.png
Since my colorbar-testimage had 313 pixel linewidth and because every pixel was encoded rgb565, I got 626 bytes per line.
But 626 is not a multiple of 4. That's why I should have added another 2 "padding bytes" to the end of each line. And that was the reason of my diagonal stripes.
After adding these 2 padding bytes (0x00 0x00), I ended up with an Bitmap Image where the header tells you: this image has a width of 313 pixels, but the real image data consists of 314 pixels per line - it's a bit weird, but that's defined by the spec.
As soon as I modified my Bitmap to comply with this requirement of the specification, the diagonal stripes disappeared and the expected vertical striped turned out of the dark.
Since 99% of all sample code in the internet assumes multiple of 4 linewidth for their images (e.g. imageformats of 320x240 or 680x480), they all do not face my problem - but most of them will, if you feed them rgb565 images with odd number of line-pixels as I had to do.
A few additional lines (marked with "// ***") were sufficient to add the "padding trick".
(The code below is just for explaining purposes, in productive code you might want to add some optimizations)
MemoryStream memoryStream = new MemoryStream(1000000);
// Read raw image into byte array
string imgpath = "rgb565_LSB-first_313x240.raw";
FileStream fs = new FileStream(imgpath, FileMode.Open);
fs.CopyTo(memoryStream);
Byte[] buffer = memoryStream.ToArray();
// Create empty Bitmep and inject byte arrays data into bitmap's data area
Bitmap bmp = new Bitmap(313, 240, PixelFormat.Format16bppRgb565);
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, 313, 240);
BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format16bppRgb565);
IntPtr ptrToFirstPixel = bmpData.Scan0;
// *** Attention: Bitmap specification requires, to pad row size to a multiple of 4 Bytes
// *** See: https://upload.wikimedia.org/wikipedia/commons/c/c4/BMPfileFormat.png
// *** Solution: Copy buffer[] to buffer2[] and pay attention to padding (!!) at the end of each row
Byte[] buffer2 = new Byte[240 * bmpData.Stride];
for (int y = 0; y < 240; y++)
{
Buffer.BlockCopy(buffer, y * 313 * 2, buffer2, y * bmpData.Stride, 313 * 2);
}
Marshal.Copy(buffer2, 0, ptrToFirstPixel, buffer2.Length); // *** Use padded buffer2 instead of buffer1
bmp.UnlockBits(bmpData);
// Diplay Bitmap in my PictureBox
pbImage.Image = bmp;

Extracting content .img file

I am working on a project where we have to extract information from .img file. What is known, is that .img file contains image with 512x512 pixels, and size of each pixel is 2 bits of type short.
We have to extract that image form the file. The question is, how to read that file with C#? My current line for binary reading is:
byte[] bytes = System.IO.File.ReadAllBytes("C:\temp\Jonatan\test23.img");
Thank you for your help!
Depending on the actual format of your pixel data, creating a bitmap from the byte array might be as simple as this:
var width = 512;
var height = 512;
var stride = width * 2;
var bitmap = BitmapSource.Create(
width, height, 96, 96, PixelFormats.Gray16, null, bytes, stride);
You might now have an Image element in XAML
<Image x:Name="image"/>
and set its Source property to the BitmapSource:
image.Source = bitmap;

Crop image using memcpy fails on 24bpp image

I’m trying to crop a 24bpp image using memcpy like I read here: cropping an area from BitmapData with C#. The problem I’m having is that it only works when my sourceImage is 32bpp. It gives me a corrupt image when my sourceImage is 24bpp.
class Program
{
[DllImport("msvcrt.dll", CallingConvention = CallingConvention.Cdecl)]
static unsafe extern int memcpy(byte* dest, byte* src, long count);
static void Main(string[] args)
{
var image = new Bitmap(#"C:\Users\Vincent\Desktop\CroppedScaledBitmaps\adsadas.png");
//Creates a 32bpp image - Will work eventhough I treat it as a 24bpp image in the CropBitmap method...
//Bitmap newBitmap = new Bitmap(image);
//Creates a 24bpp image - Will produce a corrupt cropped bitmap
Bitmap newBitmap = (Bitmap)image.Clone();
var croppedBitmap = CropBitmap(newBitmap, new Rectangle(0, 0, 150, 150));
croppedBitmap.Save(#"C:\Users\Vincent\Desktop\CroppedScaledBitmaps\PieceOfShit.png", ImageFormat.Png);
Console.ReadLine();
}
static public Bitmap CropBitmap(Bitmap sourceImage, Rectangle rectangle)
{
Console.WriteLine("Bits per pixel of sourceImage: {0}", Image.GetPixelFormatSize(sourceImage.PixelFormat));
var sourceBitmapdata = sourceImage.LockBits(rectangle, ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
var croppedImage = new Bitmap(rectangle.Width, rectangle.Height, PixelFormat.Format24bppRgb);
var croppedBitmapData = croppedImage.LockBits(new Rectangle(0, 0, rectangle.Width, rectangle.Height), ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
unsafe
{
byte* sourceImagePointer = (byte*)sourceBitmapdata.Scan0.ToPointer();
byte* croppedImagePointer = (byte*)croppedBitmapData.Scan0.ToPointer();
memcpy(croppedImagePointer, sourceImagePointer, croppedBitmapData.Stride * rectangle.Height);
}
sourceImage.UnlockBits(sourceBitmapdata);
croppedImage.UnlockBits(croppedBitmapData);
return croppedImage;
}
}
I’m very confused, because the only thing I’m changing is the sourceImage PixelFormat, not any of the code in the CropBitmap method. So I always call LockBits using 24bpp Pixelformat, even if the sourceImage is 32bpp.
I’ve tried different methods of calculating the number of bytes I’m copying but everything resulted in more or less the same corrupted image.
Any help is appreciated!
You are trying to copy the data as if it was one continuous block, but it isn't.
The image data is arranged in scan lines, but as you are selecting a part of the image, you don't want all the data from each scan line, you only want the data that represents the pixels that you have selected. A scan line contains the data for the pixels that you specified when you called LockBits, but also data for the pixels outside that area.
The Stride value is the difference in memory address from one scan line to the next. The Stride value may also include padding between the scan lines. Note also that the Stride value can be negative, which happens when the image data is stored upside down in memory.
You want to copy the relevant data from one line of the source image to the line in the destination image. As there can be gaps both in the source data and destination data, you can't copy the data as a single chunk of data.
You would need to loop through the lines and copy each line separately, I haven't tested this code, but something like this:
byte* sourceImagePointer = (byte*)sourceBitmapdata.Scan0.ToPointer();
byte* croppedImagePointer = (byte*)croppedBitmapData.Scan0.ToPointer();
int width = rectange.Width * 3; // for 24 bpp pixel data
for (int y = 0; y < rectangle.Height; y++) {
memcpy(croppedImagePointer, sourceImagePointer, width);
sourceImagePointer += sourceBitmapdata.Stride;
croppedImagePointer += croppedBitmapData.Stride;
}

Raw bitmap data/scan lines (mirror driver raw data)?

I'm coding a live control/remote desktop solution using DFMirage's free mirror driver. There is a C# sample on how to interface and control the mirror driver here. You would need the mirror driver installed first, of course, here. So, the concept is, the client (helper) requests a screen update, and the server (victim) sends one, using raw pixel encoding. The concept of a mirror driver eliminates the need to expensively poll for screen changes, because a mirror driver is notified of all screen drawing operations in real-time. The mirror driver receives the location and size of the update rectangle, and can simply query memory for the new pixel bytes and send them.
Should be easy, except that I don't know how to do that part where we query memory for the new pixel bytes. The sample shows how to query memory to grab the pixels of the entire screen using something with raw bitmap data and scan lines and stride and all that good stuff:
Bitmap result = new Bitmap(_bitmapWidth, _bitmapHeight, format);
Rectangle rect = new Rectangle(0, 0, _bitmapWidth, _bitmapHeight);
BitmapData bmpData = result.LockBits(rect, ImageLockMode.WriteOnly, format);
// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;
// Declare an array to hold the bytes of the bitmap.
int bytes = bmpData.Stride * _bitmapHeight;
var getChangesBuffer = (GetChangesBuffer)Marshal
.PtrToStructure(_getChangesBuffer, typeof (GetChangesBuffer));
var data = new byte[bytes];
Marshal.Copy(getChangesBuffer.UserBuffer, data, 0, bytes);
// Copy the RGB values into the bitmap.
Marshal.Copy(data, 0, ptr, bytes);
result.UnlockBits(bmpData);
return result;
This is great and works fine. The resulting Bitmap object now has the pixels of the entire screen. But if I wanted to just extract a rectangle of pixel data instead of getting the pixel data from the whole screen, how would I be able to do that? I guess this is more of a rawbitmap-scan-stride question, but I typed all of this so you might know where this is coming from. So any insight on how to get just a portion of pixel data instead of the entire screen's pixel data?
Update: Found something interesting (code portion only).
Here's a function to copy a rectangular area from some source image buffer to a Bitmap:
private static Bitmap ExtractImageRectangle(byte[] sourceBuffer, int sourceStride, PixelFormat sourcePixelFormat, Rectangle rectangle)
{
Bitmap result = new Bitmap(rectangle.Width, rectangle.Height, sourcePixelFormat);
BitmapData resultData = result.LockBits(new Rectangle(0, 0, result.Width, result.Height), ImageLockMode.WriteOnly, result.PixelFormat);
int bytesPerPixel = GetBytesPerPixel(sourcePixelFormat); // Left as an exercise for the reader
try
{
// Bounds checking omitted for brevity
for (int rowIndex = 0; rowIndex < rectangle.Height; ++rowIndex)
{
// The address of the start of this row in the destination image
IntPtr destinationLineStart = resultData.Scan0 + resultData.Stride * rowIndex;
// The index at which the current row of our rectangle starts in the source image
int sourceIndex = sourceStride * (rowIndex + rectangle.Top) + rectangle.Left * bytesPerPixel;
// Copy the row from the source to the destination
Marshal.Copy(sourceBuffer, sourceIndex, destinationLineStart, rectangle.Width * bytesPerPixel);
}
}
finally
{
result.UnlockBits(resultData);
}
return result;
}
You could then use it like this:
Rectangle roi = new Rectangle(100, 150, 200, 250);
Bitmap result = ExtractImageRectangle(getChangesBuffer.UserBuffer, getChangesBuffer.Stride, getChangesBuffer.PixelFormat, roi);
This assumes that GetChangesBuffer has properties for the stride and pixelformat of the source image buffer. It most likely hasn't, but you should have some means to determine the stride and pixel format of your input image. In your example you are assuming the stride of the input image is equal to the stride of your output image, which is a tricky assumption.

Getting color image from bitmap while image is 16bppGrayscale

Problem No1.
My own Related problem
I asked the next question here
Now the Problem No 2 is.
When i am trying to open 16 Bit (Monocrome ) images from their raw pixel data
then i am getting Error.
Because i am using PixelFormat.Format16bppGrayscale on creation of Bitmap like
Bitmap bmp = new Bitmap(Img_Width, Img_Height,PixelFormat.Format16bppGrayscale);
So googled and found Format16bppGrayscale not supported so i modifed my code like below.
PixelFormat format = PixelFormat.Format16bppRgb565;
Bitmap bmp = new Bitmap(Img_Width, Img_Height, format);
Rectangle rect = new Rectangle(0, 0, Img_Width, Img_Height);
BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadWrite, format);
Marshal.Copy(rawPixel, 0, bmpData.Scan0, rawPixel.Length);
bmp.UnlockBits(bmpData);
Amazing thing is that i am getting the image now because i change the pixelFormat.
But problem is that My monocrome (Grayscale) image look in various color.
How can i get the original appearance. I tried several grayscale method but not successful
Please give me some unsafe code.
Thanks,
BobPowell's GDI+ FAQ has a bit about greyscale. The 16 bpp in .NET just doesn't work. I have to do everything in 32 bpp and resort to external tools for conversion. Luckily (?) I get to stick with TIFF images most of the time. Also, this thread might help.
You need to change the pallet to a greyscale pallet. The default pallet for 8bppIndexed is 256 colour. You can change it like this:
ColorPalette pal = myBitmap.Palette;
for (int i = 0; i < pal.Entries.Length; i++)
pal.Entries[i] = Color.FromArgb(i, i, i);
myBitmap.Palette = pal;

Categories