In my C++ dll I am creating Mat from byte array:
BYTE * ptrImageData; //Image data is in this array passed to this function
Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);
The image is created with some gray shade not the original one.
Is this the proper way of creating Mat from byte array?
Please see code
ptrImageData is passed to the C++ dll from C# code.
C# code to pass the image data
System.Drawing.Image srcImage //Has the image
MemoryStream ms = new MemoryStream();
Marshal.FreeHGlobal(ptrImageData);
srcImage.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] imgArray = ms.ToArray();
ms.Dispose();
int size1 = Marshal.SizeOf(imgArray[0]) * imgArray.Length;
IntPtr ptrImageData = Marshal.AllocHGlobal(size1);
Marshal.Copy(imgArray, 0, ptrImageData, imgArray.Length);
//Calling C++ dll function
ProcessImage(ptrImageData, srcImage.Width, srcImage.Height);
Marshal.FreeHGlobal(ptrImageData);
The C++ code appears ok, in that this creates a matrix wrapper for the supplied image data, assuming the buffer is in the conventional RGB8 format. Note that this constructor does not copy the buffer, so the buffer must remain valid for the duration of this Mat instance (or be copied).
Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);
It appears the problem lies in Your C# code. I am not a C# developer, but I will do my best to help. You are creating a memory stream and using the JPEG codec to write a compressed version of the image into the buffer as if it were a file. But that is not the data format that cv::Mat is expecting, so you will basically see garbage (compressed data interpreted as uncompressed).
Given a System.Image.Drawing.Image instance, you can create a wrapper Bitmap object directly (or maybe use as, since it is a simple downcast). Then you can just use the Bitmap.LockBits() method tog obtain a pointer to the underlying image data.
Bitmap bmp = new Bitmap(sourceImage);
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.BitmapData bmpData =
bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite,
bmp.PixelFormat);
// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;
// Declare an array to hold the bytes of the bitmap.
int bytes = Math.Abs(bmpData.Stride) * bmp.Height;
byte[] rgbBuffer = new byte[bytes];
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbBuffer, 0, bytes);
// Do your OpenCV processing...
// ...
// Unlock the bits.
bmp.UnlockBits(bmpData);
and then you can pass the rgbBuffer to OpenCV.
I'm not convinced that the memory management in the original code is entirely correct either, but anyway the above will work provided the scope of the buffer ownership is within the lock and unlock method calls. If the image data is to outlive this code block, you will have to copy the buffer.
Be careful with your pixel formats too - you need to make sure the Image/Bitmap instance really contains RGB8 data. OpenCV's cv::Mat has various flags so you can work with a variety of in-memory image formats. But note that these are not the same as the on-disk (typically compressed) formats, such as PNG, TIFF, and so forth.
Yes, this is one way to create a Mat from a byte array. You just have to be careful that your array contains what you think it does.
The image is created with some gray shade not the original one.
So you are getting an image in newImg? What was the pixel format of the original data?
Maybe you've switched the red and blue channels. The following line will swap the channels:
cv::cvtColor(newImg,swappedImg,CV_RGB2BGR);
Here is link to docs: http://docs.opencv.org/modules/core/doc/basic_structures.html#mat-mat
In general you should take care about two things:
When you pass external data into matrix constructor, the external data is not automatically deallocated, so you should take care of it. If you want OpenCV matrix to care about memory, then you should copy matrix (you can do it in many ways, e.g. using Mat::clone or Mat::copyTo methods.
External data may not be continuous, i.e. size of row may be bigger than width multiplied by number of channels multiplied by size of data element. So you may want specify "step" as last argument of constructor. If you allocate external data manually and 100% sure that it is continuous, then you may not pass step and rely on automatic step calculation.
I am not familiar with C#, but it seems to me that you release data right after ProcessImage call. So if ProcessImage is asynchronous or somehow caches your matrix (i.e. lifetime of matrix is longer that ProcessImage call), then you should care about memory management.
Related
I have learnt how to export pixel data of an image to byte array, here is my code
void Button2Click(object sender, EventArgs e)
{
Bitmap img = new Bitmap (#"24x30.bmp");
var BitmapData = img.LockBits( new Rectangle(0,0,img.Width,img.Height),ImageLockMode.ReadOnly,img.PixelFormat);
var length = BitmapData.Stride * BitmapData.Height;
MessageBox.Show(BitmapData.Width.ToString());
byte[] bytes = new byte[length];
Marshal.Copy(BitmapData.Scan0, bytes, 0, length);
img.UnlockBits(BitmapData);
string test = ByteArrayToBinary(bytes);
}
I convert the bytes to string bit but lets ignore it. What I want to know is, how can I convert the byte of pixel data to an image? Please share the code and the reference.
I have read many references but I don't get it until now.
EDIT:
This summary of my case, i have Stride, Width, Height, and Byte[] of Pixel data. How can i reconstruct it to image again thanks
Same code, but copy the other way. (You can read a dummy image or use another image constructor.)
Obviously, you first step should be to somehow convert your text back to a byte array, but then you'll see that you can't actually create an image from just that data.
As you mentioned in comments already, your dumped binary block is missing all header data. You need the original width, height and pixel format before you can convert the data back to an image.
Also, if this is an indexed image, there will be a colour palette, and you don't save that either.
And finally, you copy all data in the image memory. Images are kept in memory per line of pixels, but these lines are usually rounded to the next multiple of four bytes (except in some indexed pixel formats, I think). This means that unless your image uses four bytes per pixel (32 bits per pixel), the bytes you end up with may contain junk data at the end of each line. You don't trim that off in any way, meaning you not only need the width and height, but the stride as well, before you can reconstruct the image.
As for how to build the image, the method is pretty much the same. You make a new image with the correct dimensions and pixel format, open its backing memory using LockBits but in WriteOnly mode, and copy your data into it.
I posted a full method for that on this site before, so feel free to check it out.
I'm using Microsoft Surface 2.0 SDK with SUR40 PixelSense compatible computer. I need to capture image from it's touch and save it as .bmp. Since Surface SDK comes with RawImageVisualizer example, which displays picture from touch on the screen, I've tried to modify program for writing picture to HDD. The problem is, I get ArgumentException: Parameter is invalid during building Image from byte array captured from touch.
This is how I retrieve byte array with image data from FrameReceivedEventArgs on FrameReceived event:
event.UpdateRawImage(
ImageType.Normalized,
normalizedImage,
0, 0,
InteractiveSurface.PrimarySurfaceDevice.WorkingAreaWidth,
InteractiveSurface.PrimarySurfaceDevice.WorkingAreaHeight);
And that's how I try to write bytes as .bmp to disk:
System.Drawing.Image img;
using (System.Drawing.Image raw = System.Drawing.Image.FromStream(new MemoryStream(normalizedImage)))
{
img = raw.Clone() as System.Drawing.Bitmap;
}
img.Save("C:/img.bmp", System.Drawing.Imaging.ImageFormat.Bmp);
So I get the exception trying to create Image from stream. Nevertheless this byte array works totally fine with Texture2D and SpriteBatch which displays it. How can I fix ArgumentException?
i've just realized, that UpdateRawImage does not return a byte representation of PNG file, but only an array of pixels. So, to build an image from it, one have to write all other parts of file structure to the array: header and color table (if needed). In many cases this can be simply done with one of System.Drawing.Bitmap constructors:
public Bitmap(
int width,
int height,
int stride,
PixelFormat format,
IntPtr scan0
)
But I was not so lucky, because UpdateRawImage returns 8bpp grayscale pixels, and PixelFormat enum doesn't support them (the most close is Format16bppGrayScale, but it uses 2 bytes for pixel, not one). So, in this particular situation, there are two obvious solutions. The first is making a new array of pixels, which meets one of PixelFormat standards (that was my choice, because I need 24-bit RGB image, despite it's actually black-white with only 256 shades). The second is writing BMP headers manually (and it's not very difficult due to open specs).
I have several (~2GB) raw 24bpp RGB files on HDD.
Now I want to retrieve a portion of it and scale it to the desired size.
(The only scales allowed are 1, 1/2, 1/4, 1/8, ..., 1/256)
So I'm currently reading every line from the rectangle of interest into an array, which leaves me with a bitmap which has correct height but wrong width.
As the next step I'm creating a Bitmap from the newly created array.
This is done with by using a pointer so there is no copying of data involved.
Next I'm calling GetThumbnailImage on the Bitmap, which creates a new bitmap with the correct dimensions.
Now I want to return the raw pixel data (as a byte array) of the newly created bitmap.
But to achieve that I'm currently copying the data using LockBits into a new array.
So my question is: Is there a way to get the pixel data from a Bitmap into a byte array without copying it?
Something like:
var bitmapData = scaledBitmap.LockBits(...)
byte[] rawBitmapData = (byte[])bitmapData.Scan0.ToPointer()
scaledBitmap.UnlockBits(bitmapData)
return rawBitmapData
I'm well aware that this doesn't work, it is just an example to what I basically want to achieve.
I think this is your best bet.
var bitmapData = scaledBitmap.LockBits(...);
var length = bitmapData.Stride * bitmapData.Height;
byte[] bytes = new byte[length];
// Copy bitmap to byte[]
Marshal.Copy(bitmapData.Scan0, bytes, 0, length);
scaledBitmap.UnlockBits(bitmapData);
You have to copy it, if you want a pass around a byte[].
You don't have to delete the bytes that were allocated, you just need to Dispose of the original Bitmap object when done as it implements IDisposable.
If I have a .Net Bitmap, I can create from it a GDI bitmap by calling the Bitmap's GetHbitmap() method.
Bitmap bmp = new Bitmap(100, 100);
IntPtr gdiBmp = bmp.GetHbitmap();
This works fine, but every time you call GetHbitmap, Windows has to allocate the new memory that the returned IntPtr references.
What I'd like to do - if possible - is write a function (I know PInvoke will be necessary here) that also generates a GDI bitmap copy of a Bitmap, but that overwrites an existing chunk of memory already referenced by an IntPtr returned from GetHbitmap instead of allocating new memory. So it would look something like this (if it were an extension method of Bitmap):
// desired method signature:
void OverwriteHbitmap(IntPtr gdi)
{
}
// ex:
Bitmap bmp1 = new Bitmap(100, 100);
IntPtr gdi1 = bmp1.GetHbitmap();
Bitmap bmp2 = new Bitmap(100, 100);
bmp2.OverwriteHbitmap(gdi1); // gdi1 is still pointing to the same block
// of memory, which now contains the pixel data from bmp2
How can I do this? I assume I'll need to know the structure of a GDI bitmap, and probably I can use LockBits and BitmapData for this, but I'm not sure exactly how.
Clues for the bounty hunters:
Bitmap has a method LockBits which locks the bitmap in memory and returns a BitmapData object. The BitmapData object has a Scan0 property which is an IntPtr pointing to the start of the locked bitmap's pixel data (i.e it doesn't point to the bitmap's header a.k.a. the start of the bitmap itself).
I'm pretty sure the solution looks something like this:
Bitmap bmp1 = new Bitmap(100, 100);
IntPtr gdi1 = bmp1.GetHbitmap(); // now we have a pointer to a
// 100x100 GDI bitmap
Bitmap bmp2 = new Bitmap(100, 100);
BitmapData data = bmp2.LockBits();
IntPtr gdi1Data = gdi1 + 68; // magic number = whatever the size
// of a GDI bitmap header is
CopyMemory(data.Scan0, gdi1Data, 40000);
The solution does not have to be generic - it only needs to work for bitmaps with pixel format Format32bppArgb (the default GDI+ format).
I think you answered your own question: use LockBits.
See
How to: Use LockBits
and
Bob Powell - Locking Bits.
I used this method to rapidly draw fractals in a .NET (1.1!) program that I was writing for fun. Of all the methods that I experimented with, this was by far the fastest.
Create a Graphics from the IntPtr with Graphics.FromHdc and use Graphics.DrawImage to paste the Bitmap into it.
You should not assume that a HANDLE or HBITMAP is a pointer to anything. It can be implemented as an index into a handle table, key of a hash table, etc.
However, if you call the GDI GetObject function, the resultant BITMAP structure contains a pointer to the real bits.
Not sure if it would be as fast (I think in the case of the same pixel format for source and destination it will), but SetDIBits should do exactly what you want (replace the data in an existing HBITMAP).
I am messing around with Conway's Game of Life - http://en.wikipedia.org/wiki/Conway's_Game_of_Life
I started out coding algorithmns for winforms and now want to port my work onto windows mobile 6.1 (compact framework). I came across an article by Jon Skeet where he compared several different algorithmns for calculating next generations in the game. He used an array of bytes to store a cells state (alive or dead) and then he would copy this array to an 8bpp bitmap. For each new generation, he works out the state of each byte, then copies the array to a bitmap, then draws that bitmap to a picturebox.
void CreateInitialImage()
{
bitmap = new Bitmap(Width, Height, PixelFormat.Format8bppIndexed);
ColorPalette palette = bitmap.Palette;
palette.Entries[0] = Color.Black;
palette.Entries[1] = Color.White;
bitmap.Palette = palette;
}
public Image Render()
{
Rectangle rect = new Rectangle(0, 0, Width, Height);
BitmapData bmpData = bitmap.LockBits(rect, ImageLockMode.ReadWrite, bitmap.PixelFormat);
Marshal.Copy(Data, 0, bmpData.Scan0, Data.Length);
bitmap.UnlockBits(bmpData);
return bitmap;
}
His code above is beautifully simple and very fast to render. Jon is using Windows Forms but now I want to port my own version of this onto Windows Mobile 6.1 (Compact Framework) but . . . .there is no way to format a bitmap to 8bpp in the cf.
Can anyone suggest a way of rendering an array of bytes to a drawable image in the CF. This array is created in code on the fly (it is NOT loaded from an image file on disk). I basically need to store an array of cells represented by bytes, they are either alive or dead and I then need to draw that array as an image. The game is particularly slow on the CF so I need to implement clever optimised algoritmns but also need to render as fast as possible and the above solution would be pretty dam perfect if only it was available on the compact framework.
Many thanks for any help
Any suggestions?
You could have a look at GDI+ for CF. It's basically a wrapper for most of the GDI implemented in WinCE. Here's a link to the source code and a writeup: http://community.opennetcf.com/articles/cf/archive/2007/10/31/using-gdi-on-windows-mobile.aspx
I think ImagingFactoryClass.CreateBitmapFromBuffer() looks like a good place to start.
Ok, how about this:
use the Bitmap.Save() method to save to a MemoryStream instead of a file;
when you save to the MemoryStream, you get to name the ImageFormat as "GIF" (this is equivalent to 8bpp in .Net, according to this: http://support.microsoft.com/kb/318343)
use MemoryStream.Write() to change whatever data you want in the image, or copy the data using MemoryStream.ToArray() if that jives better.
After you change the MemoryStream, you'll probably have to copy it back into the Bitmap, or make a new Bitmap. If you do make a new Bitmap, be sure to Dispose() the old one, to avoid memory leaks.
Hi Rocjoe and thanks again for the help, I have tried the following
Image bmp = new Bitmap(10, 10);
byte[] array = ImageToByteArray(bmp);
public byte[] ImageToByteArray(Image img)
{
MemoryStream ms = new MemoryStream();
img.Save(ms, System.Drawing.Imaging.ImageFormat.Gif );
return ms.ToArray();
}
The array coming back has over 870 bytes in it, It seems to hold all sorts of header info, padding and what have you. so again it does not work...