On Windows Phone 8, I wish to take a camera shot in native code, but I'm blocked on the final stage not being able to extract information from IOutputStream.
in C# we code:
MemoryStream image = new MemoryStream();
MemoryStream imagePreview = new MemoryStream();
cameraCaptureSequence.Frames[0].CaptureStream = image.AsOutputStream();
cameraCaptureSequence.Frames[0].ThumbnailStream = imagePreview.AsOutputStream();
await cameraCaptureSequence.StartCaptureAsync();
from now image stream has information of captured image and I can render it.
In C++ / Cx I need to do the same thing but more until to catch the byte* of captured image, here my code:
Windows::Phone::Media::Capture::CameraCaptureSequence^ cameraCaptureSequence;
IBuffer^ image;
return concurrency::create_async([this]()
{
cameraCaptureSequence->Frames->GetAt(0)->CaptureStream = reinterpret_cast<IOutputStream^>(image);
create_task( m_camera->PrepareCaptureSequenceAsync(cameraCaptureSequence) ).wait();
create_task( cameraCaptureSequence->StartCaptureAsync() ).then([this]()
{
}
}
Starting from the most basic thing I wish to understand how to "save" into an IBuffer^ the result of captured image stream, better how to get the internal byte* buffer.
Thanks
You can access the pixel data from a captured image in Native code through the ICameraCaptureFrameNative. The object that implements the interface is obtained through COM. Once you have obtained the object, use MapBuffer() to access the BYTE * array.
Note that the pixel data obtained this way is in NV12 format, not a JPEG or RGB as one would expect.
#include <Windows.Phone.Media.Capture.Native.h>
CameraCaptureFrame^ frame = m_cameraCaptureSequence->Frames->GetAt(0);
pNativeFrame = NULL;
HRESULT hr = reinterpret_cast<IUnknown*>(frame)->QueryInterface(__uuidof(ICameraCaptureFrameNative ), (void**) &pNativeFrame);
create_task( m_camera->PrepareCaptureSequenceAsync(m_cameraCaptureSequence) ).wait();
create_task( m_cameraCaptureSequence->StartCaptureAsync() ).then([this]()
{
DWORD bufferSize =0;
BYTE * pBuffer = NULL;
pNativeFrame->MapBuffer(&bufferSize, &pBuffer); // Pixels are in pBuffer.
// Unmap() the buffer before capturing another image.
ICameraCaptureFrameNative doesn't give access to a texture containing the preview?
If you want acces data from a IBuffer look here : http://msdn.microsoft.com/en-us/library/windows/apps/dn182761.aspx
For your case, i thinks you need a class which implement IOutputStream. Maybe InMemoryRandomAccessStream ?
Related
I have a few (8-10) large texture atlases (around 8192x8192 saved as BC1 DDS) but they are rather small disk space wise, only 32MiB.
The problem is, that when I load those files using WIC, they lose their block compression and take up to 256MiB of RAM, which shouldn't happen according to the sources
Here is my C# SharpDX code which loads the file:
public Bitmap LoadBitmap(string path) {
Bitmap m;
using (BitmapDecoder d = new BitmapDecoder(imagingFactory, $"plugins/HDPatch/{path}", Guid.Empty, SharpDX.IO.NativeFileAccess.Read, DecodeOptions.CacheOnDemand)) {
DdsDecoder decoder = d.QueryInterface<DdsDecoder>();
decoder.GetFrame(0, 0, 0, out BitmapFrameDecode frame);
m = SharpDX.Direct2D1.Bitmap.FromWicBitmap(_target, frame);//_target.CreateBitmap(converter, new BitmapProperties());
decoder.Dispose();
frame.Dispose();
}
return m;
}
Do I need to change something, or should I use a different method of loading them than WIC?
I looked at the DdsFrameDecode.CopyBlocks (+ ID2D1Bitmap::CopyFromMemory) method but I do not know how that works.. (What exactly is my stride and where do I get that information from)
EDIT:
So the BitmapFrameDecode gives me a PixelFormat of 32bppPBGRA
If I query a DDSFrameDecode from that BitmapFrameDecode, I get a format of BC1_UNorm from the DdsFrameDecode. The DdsDecoder also reports a texture with DdsAlphaModePremultiplied, and a DxgiFormat of BC1_UNorm.
But when I force the pixel format of BC1_UNorm in the BitmapProperties I get the D2D DEBUG ERROR: "The pixel format passed to this API is not compatible with the pixel format of the IWICBitmapSource"
EDIT2:
I figured out how to use CopyBlocks
SharpDX.DataStream stream = new
SharpDX.DataStream(test2.FormatInfo.BytesPerBlock * test2.SizeInBlocks.Width * test2.SizeInBlocks.Height, true, true);
test2.CopyBlocks(null, test2.FormatInfo.BytesPerBlock * test2.SizeInBlocks.Width, stream);
m = new Bitmap(_target, new SharpDX.Size2(decoder.Parameters.Width, decoder.Parameters.Height), stream, test2.FormatInfo.BytesPerBlock * test2.FormatInfo.BlockWidth, new BitmapProperties(new PixelFormat(SharpDX.DXGI.Format.BC1_UNorm, AlphaMode.Premultiplied)));
But the problem still exists. It can successfully create the Bitmap but it still consumes to much memory (250MiB+ instead of the actual block compressed size of 32MiB)
Does it maybe have to do with the render target of?
I use a software B8G8R8A8_UNorm DeviceContext render target
EDIT3:
Example project: https://drive.google.com/file/d/1hCgQnEV9xcTevgdswAiPJNREk1oDgqDC/view?usp=sharing
In my windows universal app, I'm trying to use a WinRT component: http://blogs.msdn.com/b/eternalcoding/archive/2013/03/06/developing-a-winrt-component-to-create-a-video-file-using-media-foundation.aspx (which is basically a C++ wrapper for sinkWriter)
to create a video with frames.
I put all this code in a C++ project and I can call it from my C# code without problem.
The problem come with the constructor first:
HRESULT CVideoGenerator::InitializeSinkWriter(Windows::Storage::Streams::IRandomAccessStream^ stream)
I'm not sure of how to create the stream:
var filename = "exportedVideo.wmv";
var folder = Windows.Storage.KnownFolders.VideosLibrary;
StorageFile storageFile = await folder.CreateFileAsync(filename, CreationCollisionOption.ReplaceExisting);
IRandomAccessStream stream = await storageFile.OpenAsync(FileAccessMode.ReadWrite);
StorageFile file = await StorageFile.GetFileFromPathAsync(App.PhotoModel.Path);
CVideoGenerator videoGenerator = new CVideoGenerator(1280, 720, stream, 20);
The other thing is coming from this line:
hr = sinkWriter->SetInputMediaType(streamIndex, mediaTypeIn, NULL);
//hr 0xc00d5212 : No suitable transform was found to encode or decode the content. HRESULT
Any ideas ?
I've used this VideoGenerator sample, and got the same problem.
I'm not an expert of Media Foundation, but after some researchs, I've found that the problem was at these lines :
encodingFormat = MFVideoFormat_WMV3;
inputFormat = MFVideoFormat_RGB32;
Well, I've replaced the first format by the second one, like this :
encodingFormat = MFVideoFormat_RGB32;
inputFormat = MFVideoFormat_RGB32;
And it seems to work till the new exception in the WriteSample methods
hr = MFCopyImage(
pData, // Destination buffer.
cbWidth, // Destination stride.
(BYTE*)videoFrameBuffer, // First row in source image.
cbWidth, // Source stride.
cbWidth, // Image width in bytes.
videoHeight // Image height in pixels.
);
Apparently an Access Violation while writing in the memory.
Still trying to figure it out !
McSime
I have a set of JPG images that are actually slices of a CT scan, which I want to reconstruct into DICOM image files and import into a PACS.
I am using ClearCanvas, and have set all of the requisite tags (and confirmed them by converting one of my JPG files to DICOM using a proprietary application to make sure they are the same). I am just not sure how I should be processing my JPG file to get it into the PixelData tag?
Currently I am converting it to a Byte array, on advice from ClearCanvas forums, but the image is just garbled in the DICOM viewer. How should I be processing the image data to get it into a readable format?
public DicomFile CreateFileFromImage(Image image)
{
int height = image.Height;
int width = image.Width;
short bitsPerPixel = (short)Image.GetPixelFormatSize(image.PixelFormat);
byte[] imageBuffer = ImageToByteArray(image);
DicomFile dicomFile = new DicomFile();
dicomFile.DataSet[DicomTags.Columns].SetInt32(0, width);
dicomFile.DataSet[DicomTags.Rows].SetInt32(0, height);
dicomFile.DataSet[DicomTags.BitsStored].SetInt16(0, bitsPerPixel);
dicomFile.DataSet[DicomTags.BitsAllocated].SetInt16(0, bitsPerPixel);
dicomFile.DataSet[DicomTags.HighBit].SetInt16(0, 7);
//other tags not shown
dicomFile.DataSet[DicomTags.PixelData].Values = imageBuffer;
return dicomFile;
}
public static byte[] ImageToByteArray(Image imageIn)
{
MemoryStream ms = new MemoryStream();
imageIn.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
return ms.ToArray();
}
The ClearCanvas library as two helper classes that make it easier to encode and decode pixel data within a DicomFile. They are the DicomCompressedPixelData class and the DicomUncompressedPixelData class. You can use these to set the parameters for the image, and encode them into the DicomFile object.
In your case, since you're encoding a compressed object, you should use the DicomCompressedPixelData class. There are properties on the class that can be set. Calling the UpdateMessage method will copy these property values over to the DicomFile object. Also, this class has an AddFrameFragment method that properly encodes the pixel data. Note that compressed pixel data has to have some specific binary wrappers around each frame of data. This was the part missing from your previous code. The code below shows how to set this up.
short bitsPerPixel = (short)Image.GetPixelFormatSize(image.PixelFormat);
var dicomFile = new DicomFile();
var pd = new DicomCompressedPixelData(dicomFile);
pd.ImageWidth = (ushort)image.Width;
pd.ImageHeight = (ushort) image.Height;
pd.BitsStored = (ushort)bitsPerPixel;
pd.BitsAllocated = (ushort) bitsPerPixel;
pd.HighBit = 7;
pd.SamplesPerPixel = 3;
pd.PlanarConfiguration = 0;
pd.PhotometricInterpretation = "YBR_FULL_422";
byte[] imageBuffer = ImageToByteArray(image);
pd.AddFrameFragment(imageBuffer);
pd.UpdateMessage(dicomFile);
return dicomFile;
I ended up processing the bitmap manually and creating an array out of the Red Channel in the image, following some code in a plugin:
int size = rows * columns;
byte[] pData = new byte[size];
int i = 0;
for (int row = 0; row < rows; ++row)
{
for (int column = 0; column < columns; column++)
{
pData[i++] = image.GetPixel(column, row).R;
}
}
It does work, but it is horribly slow and creates bloated DICOM files. I'd love to get the inbuilt DicomCompressedPixelData class working.
Any further suggestions would be very welcome.
It is important to know the bit depth and color components of your JPEG CT image before inserting into DICOM dataset. It could be 8-bit lossy (JPEG Compression Process 2) or 12-bit lossy (JPEG Compression Process 4) or 8, 12 or 16 bit lossless grayscale JPEG (JPEG Compression Process 14 - lossless, non-hierarchical). This information is critical for updating the Pixel Data related information such as Photometric Interpretation, Sample per Pixel, Planer Configuration, Bits Allocated, High Bit to Transfer Syntax.
I make a webrequest to receive a large jpeg as a byte array. This in turn can be converted to a memory stream. I need to get this data into a bitmapdata so that I can marshall copy it to a byte array again. Am i right in assuming that a byte array returned from a memory stream is not the same as a byte array returned from a marshall copy of bitmapdata to a byte array?
I do not want to write the memory stream out to an image as it will return a out of memory error due to its size AND the fact I am using compact cf C# 2.
this is my call to the server..
HttpWebRequest _request = (HttpWebRequest)WebRequest.Create("A url/00249.jpg");
_request.Method = "GET";
_request.Timeout = 5000;
_request.ReadWriteTimeout = 20000;
byte[] _buffer;
int _blockLength = 1024;
int _bytesRead = 0;
MemoryStream _ms = new MemoryStream();
using (Stream _response = ((HttpWebResponse)_request.GetResponse()).GetResponseStream())
{
do
{
_buffer = new byte[_blockLength];
_bytesRead = _response.Read(_buffer, 0, _blockLength);
_ms.Write(_buffer, 0, _bytesRead);
} while (_bytesRead > 0);
}
This is my code to read a byte array from a bitmapdata.
Bitmap Sprite = new Bitmap(_file);
Bitmapdata RawOriginal = Sprite.LockBits(new Rectangle(0, 0, Sprite.Width, Sprite.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppRgb);
int origByteCount = RawOriginal.Stride * RawOriginal.Height;
SpriteBytes = new Byte[origByteCount];
System.Runtime.InteropServices.Marshal.Copy(RawOriginal.Scan0, SpriteBytes, 0, origByteCount);
Sprite.UnlockBits(RawOriginal);
Note:
I do not want to use this:
Bitmap Sprite = new Bitmap(_file);
I want to go from:
MemoryStream _ms = new MemoryStream();
to
System.Runtime.InteropServices.Marshal.Copy(RawOriginal.Scan0, SpriteBytes, 0, origByteCount);
using what ever conversions are required without writing to a bitmap.
What you're asking is going to be difficult. The data you're receiving from the response object is a full jpeg image, which has a header and then a bunch of compressed data bytes. The byte array addressed by Scan0 is uncompressed and quite possibly includes some padding bytes at the end of each scan line.
Most importantly, you definitely cannot use Marshal.Copy to copy the received bytes to Scan0.
To do what you're asking will require that you parse the header of the jpeg that you receive and uncompress the image bits directly to Scan0, padding each scan line as appropriate. There is nothing in the .NET Framework that will do that for you.
The accepted answer to this question has a link to a library that might help you out.
Even if that works, I'm not certain it will help you out. If calling the BitMap constructor to create the image causes you to run out of memory, it's almost certain that this roundabout method will, as well.
Is the problem that you have so many sprites that you can't keep them all in memory, uncompressed? If so, you'll probably have to find some other way to solve your problem.
By the way, you can save yourself a lot of trouble by changing your code that reads the image to:
MemoryStream _ms = new MemoryStream();
using (Stream _response = ((HttpWebResponse)_request.GetResponse()).GetResponseStream())
{
_response.CopyTo(_ms);
}
I'm running into an issue where I'm trying to copy the pixel buffer for one WriteableBitmap over to another WriteableBitmap essentially giving a copy of the WriteableBitmap object. However, when I try to do this I run into an issue where the second WriteableBitmap's stream length is too short to hold all the values of the first WriteableBitmap.
I posted my code below. Keep in mind that I'm capturing the original data from a webcam. However, when I compare the "ps" object's stream size to wb1 and wb2, ps's size is much smaller than both of them. What I'm confused about is why wb2 stream size is smaller than wb1's. Thanks for any help.
private MemoryStream originalStream = new MemoryStream();
WriteableBitmap wb1 = new WriteableBitmap((int)photoBox.Width, (int)photoBox.Height);
WriteableBitmap wb2 = new WriteableBitmap((int)photoBox.Width, (int)photoBox.Height);
ImageEncodingProperties imageProperties = ImageEncodingProperties.CreateJpeg();
var ps = new InMemoryRandomAccessStream();
await mc.CapturePhotoToStreamAsync(imageProperties, ps);
await ps.FlushAsync();
ps.Seek(0);
wb1.SetSource(ps);
(wb1.PixelBuffer.AsStream()).CopyTo(originalStream); // this works
originalStream.Position = 0;
originalStream.CopyTo(wb2.PixelBuffer.AsStream()); // this line gives me the error: "Unable to expand length of this stream beyond its capacity"
Image img = new Image();
img.Source = wb2; // my hope is to treat this as it's own entity and modify this image independently of wb1 or originalStream
photoBox.Source =wb1;
Note that when you do new WriteableBitmap(w, h) and then call SetSource() to an image of a different resolution - the bitmap's size will change (it won't be the w x h passed in the constructor). It's likely that your photoBox.Width/Height are different than what your CapturePhotoToStreamAsync() call returns (I am assuming the image is captured at the default or preconfigured camera settings, while photoBox is just a control on screen).
How about just doing someting like this
ps.Seek(0);
wb1.SetSource(ps);
ps.Seek(0);
wb2.SetSource(ps);
I think you should create a writter from the PixelBuffer and use it to copy the stream.
The AsStream method should be used to read the buffer, not to write into it.
Have a look to
http://social.msdn.microsoft.com/Forums/en-NZ/winappswithcsharp/thread/2b499ac5-8bc8-4259-a144-842bd756bfe2
for a piece of code