I am working with mediaframes (Kinect) to get colour, Depth/Infrared frames on UWP in realtime. This is to store frame data on disk and later process it.
For colour, I get pixels in bytes by using Memorystream.
// Get the Individual color Frame
var vidFrame = clrFrame?.VideoMediaFrame;
{
if (vidFrame == null) return;
// create a UWP SoftwareBitmap and copy Color Frame into Bitmap
SoftwareBitmap sbt = new SoftwareBitmap(vidFrame.SoftwareBitmap.BitmapPixelFormat, vidFrame.SoftwareBitmap.PixelWidth, vidFrame.SoftwareBitmap.PixelHeight);
vidFrame.SoftwareBitmap.CopyTo(sbt);
// PixelFormat needs to be in 8bit for Colour only
if (sbt.BitmapPixelFormat != BitmapPixelFormat.Bgra8)
sbt = SoftwareBitmap.Convert(vidFrame.SoftwareBitmap, BitmapPixelFormat.Bgra8);
if (source != null)
{
var ignore = Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
{
extBitmap = new WriteableBitmap(sbt.PixelWidth, sbt.PixelHeight);
sbt.CopyToBuffer(extBitmap.PixelBuffer);
byte[] pixels = PixelBufferToWriteableBitmap(extBitmap);
extBitmap.Invalidate();
await SavePixelsToFile(pixels);
});
}
}
public async Task<byte[]> PixelBufferToWriteableBitmap(WriteableBitmap wb)
{
using (Stream stream = wb.PixelBuffer.AsStream())
{
using (MemoryStream memoryStream = new MemoryStream())
{
await stream.CopyToAsync(memoryStream);
byte[] pixels = memoryStream.ToArray();
return pixels;
}
}
}
The Infrared pixelformat is Gray16 (in SoftwareBitmap); I want to keep the raw pixel data (so no data is lost from the frame) and write it to the localfolder in ushort[] array.
Following are the links, I came across on how to get set pixel from software bitmap. However, it is bgra to byte and I want to convert software bitmap into ushort.
How to set/get pixel from Softwarebitmap
https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/imaging
I am new at this and not sure how to proceed, with this.
Can someone please help?
EDIT
I figured that a buffer mediaframe can be converted to byte array by doing the following:
public async Task<byte[]> BufferStreamTobyte(BufferMediaFrame buffFrame)
{
using (Stream stream = buffFrame.Buffer.AsStream())
{
using (MemoryStream memoryStream = new MemoryStream())
{
await stream.CopyToAsync(memoryStream);
byte[] pixels = memoryStream.ToArray();
return pixels;
}
}
}
But I am not sure if I loose information of Infrared frame by doing this. Since Infrared and Depth are 16 bits per pixel and this current byte conversion holds 8 bpp. For this ushort[] would be able to hold 16bpp.
I very new to this and not sure so I hope I have gotten this right?
EDIT 2:
I have got the pixel data in byte[].
I understand that byte is 8 bits and short is 16 bits so I changed the length of the arrays:
int width = softwareBitmap.PixelWidth;
int height = softwareBitmap.PixelHeight;
int length = width * height * 2;
byte[] irbyteData = new byte[length]; // *2 to get 16 bit
var irshortData = new ushort[width * height]; // 16 bit ushort
IntPtr ptr = (IntPtr)pixelBytesAddress;
Marshal.Copy(ptr, irbyteData, 0, length); //seems to successfully get pixels from memory but not sure if this is lossless
I would try using PNGs
ex:
using(var fs=new FileStream("sample.png"))
{
BitmapSource bmpSource=BitmapSource(Width,Height,96,96,PixelFormats.Gray16,
BitmapPalettes.Gray16,buffer,ImageWidth*sizeof(ushort));
PngBitmapEncoder enc = new PngBitmapEncoder();
end.Frames.Add(BitmapFrame.Create(bmpSource));
enc.Save(fs)
}
Sorry if there's a typo in a code I write from memory on a computer without IDE
Add System.Windows.media.Imaging to your usings
buffer is a ushort array with the pixel values (should be in total length: width*height*2)
I found a solution to copy IntPtr data to ushort array through the following link:
Copy from IntPtr (16 bit) array to managed ushort
I get the address IntPtr by
using (var input = softwareBitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
using (var inputReference = input.CreateReference())
((IMemoryBufferByteAccess)inputReference).GetBuffer(out inputBytes, out inputCapacity);
IntPtr ptr = (IntPtr)inputBytes;
Marshal.Copy(ptr, infraredbyteData, 0, length);
I get the bytes with length( width *height *2 ) to hold 16 bit data.
Later I convert it to ushort by
var size = infraredbyteData.Length / 2;
ushort[] output = new ushort[size]; // every ushort is 2 bytes
Buffer.BlockCopy(infraredbyteData, 0, output, 0, infraredbyteData.Length);
This seems to work!
Related
I am successfully converting a JPG image file into a base64 string, and then that string into a JPG image again.
string str64 = ImageToStrBase64(My.Resources.ImageJpg);
PictureBox1.Image = Base64StrToImage(str64);
ImageToStrBase64() and Base64StrToImage() are custom functions just to explain the idea, but I can place the code if necessary.
I am also converting a raw byte array (RGB or BGR, no matters) into a base64.
However, I do now need to convert a raw byte array into a JPG encoded base64 string. I am finding solutions that involves only an image file saving, but file management it's high time consuming. How to encode then a BGR byte array into a JPG byte array using, for example a memory stream?
Function use to convert jpg into a formatted byte array:
public static string ImagetoStrBase64(System.Drawing.Image imageToEncode)
{
string base64String = "";
using (System.Drawing.Image image = imageToEncode)
{
using (MemoryStream m = new MemoryStream())
{
image.Save(m, image.RawFormat);
byte[] imageBytes = m.ToArray();
base64String = Convert.ToBase64String(imageBytes);
}
}
return base64String;
}
Update (regarding to chat)
Let's simplify it... We have a byte[] with single green pixel:
byte[] rawPixelArray = new byte[] {0, 255, 0};
Then, GetJpgBytes(rawPixelArray) must return an encoded byte[] for a Jpeg image of 1x1 pixel. This is more clear I guess :)
Before you read the answer, read the bullet points:
A real useful conversion, should return JPEG encoded image which is equivalent to content of a Jpeg file, which is not pixel data.
When you get an array of pixel data of a Jpeg image, you will have non of benefits of a Jpeg image. It's no more Jpeg encoded image, it's an array of pixel data which can be used to create a BitMap.
To better understand the point, for example for a 10K JPEG white image, the pixel data will be 2MB!
I've shared a few methods in this answer which are really useful standalone, and I've showed how you can combine them to get what you want. (While I cannot see any benefit in the conversion which you are trying to do.)
These are very useful pieces which you can put together to solve the puzzle:
ImageData: In addition to a byte[] of image pixels, you need to know about width, height and pixel format of the image. Then you can describe the image and create an Image from that data. Without having all these information a byte[] is meaningless on its own.
GetImageDataFromImage: Gets image data (width, height, pixel format, pixel data) from an Image
CreateImageFromImageData: Creates an Image from image data
ConvertImageToJpegImage: Converts an Image to Jpeg Image
ByteArrayToBase64: Converts a byte[] to Base64 string.
Then after having these pieces, you can achieve what you want.
Assuming you have a byte[] of pixel data, width, height and pixel format of your data, this is what you need:
// A 2x2 image having 24 bit RGB pixel format, all pixels are #FF0000 (Red)
var imageData = new ImageData()
{
Width = 2,
Height = 2,
PixelFormat = PixelFormat.Format24bppRgb,
PixelData = new byte[] {
0x0, 0x0, 0xFF, 0x0, 0x0, 0xFF,
0xF, 0x0, 0xFF, 0x0, 0x0, 0xFF
}
};
var image = CreateImageFromImageData(imageData);
var jpeg = ConvertImageToJpegImage(image);
var jpegImageData = GetImageDataFromImage(jpeg);
var jpegPixelsBase64 = ByteArrayToBase64(jpegImageData.PixelData);
Which result in AAD+AAD+AAD+AAD+ which is in fact an image having #FE0000 color!
Note: I didn't dispose images to keep it clear, in action, you need to dispose image and jpeg.
ImageData
Required information about an image, including width, height, pixel format, and pixel data array:
public class ImageData
{
public int Width { get; set; }
public int Height { get; set; }
public PixelFormat PixelFormat { get; set; }
public byte[] PixelData { get; set; }
}
GetImageDataFromImage
Gets image data from an image.
ImageData GetImageDataFromImage(Image image)
{
using (var bitmap = new Bitmap(image.Width, image.Height, image.PixelFormat))
{
using (var g = Graphics.FromImage(bitmap))
g.DrawImage(image, new Rectangle(0, 0, image.Width, image.Height));
var data = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height),
ImageLockMode.ReadOnly, bitmap.PixelFormat);
var rowLength = image.Width * Image.GetPixelFormatSize(image.PixelFormat) / 8;
var bytes = new byte[image.Height * rowLength];
var ptr = data.Scan0;
for (var i = 0; i < image.Height; i++)
{
Marshal.Copy(ptr, bytes, i * rowLength, rowLength);
ptr += data.Stride;
}
bitmap.UnlockBits(data);
return new ImageData
{
Width = bitmap.Width,
Height = bitmap.Height,
PixelFormat = bitmap.PixelFormat,
PixelData = bytes
};
}
}
We rely on LockBits to get or set image byte array.
CreateImageFromImageData
Creates image from image data:
Image CreateImageFromImageData(ImageData imageData)
{
var bitmap = new Bitmap(imageData.Width, imageData.Height, imageData.PixelFormat);
var data = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height),
ImageLockMode.ReadWrite, bitmap.PixelFormat);
var rowLength = imageData.Width * Image.GetPixelFormatSize(imageData.PixelFormat) / 8;
var bytes = new byte[imageData.Height * rowLength];
var ptr = data.Scan0;
for (var i = 0; i < imageData.Height; i++)
{
Marshal.Copy(imageData.PixelData, i * rowLength, ptr, rowLength);
ptr += data.Stride;
}
bitmap.UnlockBits(data);
return bitmap;
}
We rely on LockBits to get or set image byte array.
ConvertImageToJpegImage
Converts an image to Jpeg Image:
public Image ConvertImageToJpegImage(Image img)
{
using (var stream = new MemoryStream())
{
img.Save(stream, ImageFormat.Jpeg);
var bytes = stream.ToArray();
return (Image)new ImageConverter().ConvertFrom(bytes);
}
}
If you care about compression level use jpeg encoder.
ByteArrayToBase64
Conversion from byte[] to Base64String and is straightforward, but to have better readability of the answer and the code:
public string ByteArrayToBase64(byte[] bytes)
{
return Convert.ToBase64String(bytes);
}
I'm having some trouble converting an image to a video using the SharpAVI.dll.
I have managed to produce a video file using a randomly generated byte array by using the documentation on SharpAVI's website:
Getting Started with SharpAVI
So the next step I thought I would take was to take an Image, create a Bitmap image, convert the bitmap to a byte array and then simply save the byte array to each frame of the video file. When I run the program, I get no errors or anything and a video file of an appropriate file size is produced however the video file is unreadable and will not open. I'm really struggling to see why this won't work. Any help would be greatly appreciated!
My Code:
private void GenerateSingleImageVideo()
{
string imagePath = textBoxImagePath.Text;
Bitmap thisBitmap;
//generate bitmap from image file
using (Stream BitmapStream = System.IO.File.Open(imagePath, FileMode.Open))
{
Image img = Image.FromStream(BitmapStream);
thisBitmap = new Bitmap(img);
}
//convert the bitmap to a byte array
byte[] byteArray = BitmapToByteArray(thisBitmap);
//creates the writer of the file (to save the video)
var writer = new AviWriter(textBoxFileName.Text + ".avi")
{
FramesPerSecond = int.Parse(textBoxFrameRate.Text),
EmitIndex1 = true
};
var stream = writer.AddVideoStream();
stream.Width = thisBitmap.Width;
stream.Height = thisBitmap.Height;
stream.Codec = KnownFourCCs.Codecs.Uncompressed;
stream.BitsPerPixel = BitsPerPixel.Bpp32;
int numberOfFrames = ((int.Parse(textBoxFrameRate.Text)) * (int.Parse(textBoxVideoLength.Text)));
int count = 0;
while (count <= numberOfFrames)
{
stream.WriteFrame(true, byteArray, 0, byteArray.Length);
count++;
}
writer.Close();
MessageBox.Show("Done");
}
private byte[] BitmapToByteArray(Bitmap img)
{
ImageConverter converter = new ImageConverter();
return (byte[])converter.ConvertTo(img, typeof(byte[]));
}
You're wrong in assuming that you should pass a Bitmap object to WriteFrame method. It expects pixel data in bottom to top 32bpp format. See example in
// Buffer for pixel data
var buffer = new byte[width * height * 4];
...
// Copy pixels from Bitmap assuming it has expected 32bpp pixel format
var bits = bitmap.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, PixelFormat.Format32bppRgb);
Marshal.Copy(bits.Scan0, buffer, 0, buffer.Length);
bitmap.UnlockBits(bits);
You can see code of a sample app as a reference
https://github.com/baSSiLL/SharpAvi/blob/master/Sample/Recorder.cs
I am using BinaryReader to read the bytes of an image, I am having some issues trying to read the ARGB values of a bitmap image using BinaryReader. Can anyone suggest a way I could get the byte value for each pixel in a bitmap image?
Thanks in advance
Easy way is to use unsafe context and lock some bits. Oversimplified sample:
unsafe
{
var bitmapData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);
byte* first = (byte*)bitmapData.Scan0;
byte a = first[0];
byte r = first[1];
byte g = first[2];
byte b = first[3];
...
bmp.UnlockBits(bitmapData);
}
However, if you still need to use BinaryReader and you know how many bytes per pixel there are, It is possible to just skip the header (you can find it's length in #Bradley_Ufffner 's link) and access the bytes.
You will need to study the BMP file format available here: http://en.wikipedia.org/wiki/BMP_file_format
Reading the file correctly will involve figuring out the pixel format from the header and parsing the data correctly based on that. The file may be palatalized, in which case you will need to read out the color table data and use it to map pixels to actual colors. Pixel data may also be compressed and will have to be extracted based on values in the header.
This won't be a simple project, things like this are the reason graphics libraries were invented.
If you need to read a bitmap's pixel data using BinaryReader, try UnmanagedMemoryStream:
Bitmap bmp = new Bitmap("img.bmp");
var bits = bmp.LockBits(new Rectangle(0,0,bmp.Width,bmp.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
try{
unsafe{
using(Stream bmpstream = new UnmanagedMemoryStream((byte*)bits.Scan0, bits.Height*bits.Stride))
{
BinaryReader reader = new BinaryReader(bmpstream);
for(int y = 0; y < bits.Height; y++)
{
bmpstream.Seek(bits.Stride*y, SeekOrigin.Begin);
for(int x = 0; x < bits.Width; x++)
{
byte b = reader.ReadByte();
byte g = reader.ReadByte();
byte r = reader.ReadByte();
byte a = reader.ReadByte();
}
}
}
}
}finally{
bmp.UnlockBits(bits);
}
This puzzles me for last two hours. Reading an image file results in different pixel values between imread in Matlab and Image.FromFile in C#?
aa=imread('myfile.tif')
max(aa(:)) = 248 in matlab
In C#
var image2Array = imageToByteArray((Bitmap) Image.FromFile("myfile.tif"));
byte maxx = 0;
foreach(var a in image2Array)
{
maxx = Math.Max(maxx, a);
}
//maxx = 255
Futhermore, in Matlab,
aa(1,1) = 13,
aa(1,2) = 13
but in C#
image2Array[0]=17,
image2Array[1]=0
They should be the same.
BTW, in this case, pixel type is uint8. so there is no dimensional difference.
If you ask me how I got byte array from Image, I used MSDN document to make this method.
public byte[] imageToByteArray(Bitmap bmp)
{
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
BitmapData bmpData = bmp.LockBits(
rect,
ImageLockMode.ReadWrite,
bmp.PixelFormat);
// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;
// Declare an array to hold the bytes of the bitmap.
int bytes = Math.Abs(bmpData.Stride)*bmp.Height;
byte[] rgbValues = new byte[bytes];
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);
// Unlock the bits.
bmp.UnlockBits(bmpData);
return rgbValues;
}
What did I do wrong here? I suspect that they use different reading algorithms because two resulting images look same.
UPDATE:
I don't think there is anything wrong with what I was doing. I concluded that reading tif as a bitmap was the cause of the problem. To confirm this theory,
I displayed the two images and they looked exactly the same. So there is no mistake on my part, I think.
I tried to read the same file with opencv and its pixel values were exactly the same as the ones from matlab. This was surprisingly to me. I would very cautiously use Bitmap in C# from now on.
Your imageToByteArray method does return a byte array, but you can't assume each byte is a pixel. The PixelFormat determines how the pixel data is stored in the byte array.
The best site I've seen that documents this is Bob Powell's lockbits page.
If the PixelFormat is Format8bppIndexed, then this (untested) code should give you the color values for each pixel.
var bmp = (Bitmap)Bitmap.FromFile("myfile.tif");
// ******* Begin copying your imageToByteArray method
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
BitmapData bmpData = bmp.LockBits(
rect,
ImageLockMode.ReadWrite,
bmp.PixelFormat);
// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;
// Declare an array to hold the bytes of the bitmap.
int bytes = Math.Abs(bmpData.Stride) * bmp.Height;
byte[] imageData = new byte[bytes];
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, imageData, 0, bytes);
// Unlock the bits.
bmp.UnlockBits(bmpData);
// ******* End copying your imageToByteArray method
// Now loop through each pixel... The byte array contains extra bytes
// used for padding so we can't just loop through every byte in the array.
// This is done by using the Stride property on bmpData.
for (int y = 0; y < bmpData.Height; y++)
{
for (int x = 0; x < bmpData.Width; x++)
{
var offset = (y * bmpData.Stride) + x;
// The byte in the image array gives the offset into the palette
var paletteIndex = imageData[offset];
// Given the offset, find the matching color in the palette
var color = bmp.Palette.Entries[offset];
// Look at the color value here...
}
}
TIFF has many formats, you are attempting to read it as a bitmap.
I suggest reading it using a proprietary TIFF reader instead : Good Tiff library for .NET
I am writing a library to interface C# with the EPL2 printer language. One feature I would like to try to implement is printing images, the specification doc says
p1 = Width of graphic Width of graphic in bytes. Eight (8) dots = one (1) byte of data.
p2 = Length of graphic Length of graphic in dots (or print lines)
Data = Raw binary data without graphic file formatting. Data must be in bytes. Multiply the width in bytes (p1) by the number of print lines (p2) for the total amount of graphic data. The printer automatically calculates the exact size of the data block based upon this formula.
I plan on my source image being a 1 bit per pixel bmp file, already scaled to size. I just don't know how to get it from that format in to a byte[] for me to send off to the printer. I tried ImageConverter.ConvertTo(Object, Type) it succeeds but the array it outputs is not the correct size and the documentation is very lacking on how the output is formatted.
My current test code.
Bitmap i = (Bitmap)Bitmap.FromFile("test.bmp");
ImageConverter ic = new ImageConverter();
byte[] b = (byte[])ic.ConvertTo(i, typeof(byte[]));
Any help is greatly appreciated even if it is in a totally different direction.
If you just need to convert your bitmap into a byte array, try using a MemoryStream:
Check out this link: C# Image to Byte Array and Byte Array to Image Converter Class
public byte[] imageToByteArray(System.Drawing.Image imageIn)
{
MemoryStream ms = new MemoryStream();
imageIn.Save(ms,System.Drawing.Imaging.ImageFormat.Gif);
return ms.ToArray();
}
As SLaks said I needed to use LockBits
Rectangle rect = new Rectangle(0, 0, Bitmap.Width, Bitmap.Height);
System.Drawing.Imaging.BitmapData bmpData = null;
byte[] bitVaues = null;
int stride = 0;
try
{
bmpData = Bitmap.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadOnly, Bitmap.PixelFormat);
IntPtr ptr = bmpData.Scan0;
stride = bmpData.Stride;
int bytes = bmpData.Stride * Bitmap.Height;
bitVaues = new byte[bytes];
System.Runtime.InteropServices.Marshal.Copy(ptr, bitVaues, 0, bytes);
}
finally
{
if (bmpData != null)
Bitmap.UnlockBits(bmpData);
}