We Keep Coding

Application crash while Bitmap image assign to Picturebox in Winforms

I am new here.
In my application i assign Bitmap image to picturebox. But after some time my application crash. I also maintan try catch and logs but application is just crash.
Here is my code:
System.Drawing.Bitmap ImageBMP = new System.Drawing.Bitmap(ImageWidth, ImageHeight, stride, PixelFormat.Format8bppIndexed, new IntPtr(scan0));
if (Picturebox1!= null && Picturebox1.Image != null)
{
Picturebox1.Image.Dispose();
Picturebox1.InitialImage = null;
}
Picturebox1.Image =ImageBMP;
Thanks in advance.

Because you disposed the object inside if block. Remove that and it won't give error.
Picturebox1.Image.Dispose();
Hope helps,

You are using a raw pointer there. Where does that come from? It is advised to use managed arrays, unless you can be 100% sure that that pointer will remain valid.
If it comes from a LockBits operation on another image, it will not remain valid; it will stop being reliable from the moment the other image is unlocked.
If you are planning to clone or edit an 8bpp image, it is much safer to copy the contents of the images you're manipulating into normal managed Byte[] arrays, using LockBits and Marshal.Copy, and to copy them back into Bitmap objects the same way, rather than using pointers directly.
These pieces of code should set you on your way:
Get the backing byte array from an image:
/// <summary>
/// Gets the raw bytes from an image.
/// </summary>
/// <param name="sourceImage">The image to get the bytes from.</param>
/// <param name="stride">Stride of the retrieved image data.</param>
/// <returns>The raw bytes of the image</returns>
public static Byte[] GetImageData(Bitmap sourceImage, out Int32 stride)
{
BitmapData sourceData = sourceImage.LockBits(new Rectangle(0, 0, sourceImage.Width, sourceImage.Height), ImageLockMode.ReadOnly, sourceImage.PixelFormat);
stride = sourceData.Stride;
Byte[] data = new Byte[stride * sourceImage.Height];
Marshal.Copy(sourceData.Scan0, data, 0, data.Length);
sourceImage.UnlockBits(sourceData);
return data;
}
Create an image from a byte array: (rather than from a pointer)
A: Why must “stride” in the System.Drawing.Bitmap constructor be a multiple of 4?
And, combined and optimised:
Make a deep clone of an image to load it without any linked resources
A: Free file locked by new Bitmap(filePath)
Note, as for your disposing... you should never dispose an image still linked to the UI, since the next repaint of the UI will attempt to use the disposed image, which will inevitably cause a crash. The correct way to do this is to store the image reference in a variable, then make it null on the UI, and then dispose it:
if (Picturebox1 != null && Picturebox1.Image != null)
{
Image img = Picturebox1.Image;
Picturebox1.Image = null;
img.Dispose();
}

How can I update this kinect depth-basics code so that i can save raw depth data such as uint16 format in C#?

I want to change this code so it can save raw depth data as uint16 format. but at this stage, it saves the image in uint8 format by converting the values 0-255. but i need to save the value as it come from camera or kinect such as uint16 format.
Code:depthBasics:
//------------------------------------------------------------------------------
// <copyright file="MainWindow.xaml.cs" company="Microsoft">
// Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
//------------------------------------------------------------------------------
namespace Microsoft.Samples.Kinect.DepthBasics
{
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using Microsoft.Kinect;
/// <summary>
/// Interaction logic for MainWindow
/// </summary>
public partial class MainWindow : Window, INotifyPropertyChanged
{
/// <summary>
/// Map depth range to byte range
/// </summary>
private const int MapDepthToByte = 8000 ;
/// <summary>
/// Active Kinect sensor
/// </summary>
private KinectSensor kinectSensor = null;
/// <summary>
/// Reader for depth frames
/// </summary>
private DepthFrameReader depthFrameReader = null;
/// <summary>
/// Description of the data contained in the depth frame
/// </summary>
private FrameDescription depthFrameDescription = null;
/// <summary>
/// Bitmap to display
/// </summary>
private WriteableBitmap depthBitmap = null;
/// <summary>
/// Intermediate storage for frame data converted to color
/// </summary>
private byte[] depthPixels = null;
/// <summary>
/// Current status text to display
/// </summary>
private string statusText = null;
/// <summary>
/// Initializes a new instance of the MainWindow class.
/// </summary>
public MainWindow()
{
// get the kinectSensor object
this.kinectSensor = KinectSensor.GetDefault();
// open the reader for the depth frames
this.depthFrameReader = this.kinectSensor.DepthFrameSource.OpenReader();
// wire handler for frame arrival
this.depthFrameReader.FrameArrived += this.Reader_FrameArrived;
// get FrameDescription from DepthFrameSource
this.depthFrameDescription = this.kinectSensor.DepthFrameSource.FrameDescription;
// allocate space to put the pixels being received and converted
this.depthPixels = new byte[this.depthFrameDescription.Width * this.depthFrameDescription.Height];
// create the bitmap to display
this.depthBitmap = new WriteableBitmap(this.depthFrameDescription.Width, this.depthFrameDescription.Height, 96.0, 96.0, PixelFormats.Gray8, null);
// set IsAvailableChanged event notifier
this.kinectSensor.IsAvailableChanged += this.Sensor_IsAvailableChanged;
// open the sensor
this.kinectSensor.Open();
// set the status text
this.StatusText = this.kinectSensor.IsAvailable ? Properties.Resources.RunningStatusText
: Properties.Resources.NoSensorStatusText;
// use the window object as the view model in this simple example
this.DataContext = this;
// initialize the components (controls) of the window
this.InitializeComponent();
}
/// <summary>
/// INotifyPropertyChangedPropertyChanged event to allow window controls to bind to changeable data
/// </summary>
public event PropertyChangedEventHandler PropertyChanged;
/// <summary>
/// Gets the bitmap to display
/// </summary>
public ImageSource ImageSource
{
get
{
return this.depthBitmap;
}
}
/// <summary>
/// Gets or sets the current status text to display
/// </summary>
public string StatusText
{
get
{
return this.statusText;
}
set
{
if (this.statusText != value)
{
this.statusText = value;
// notify any bound elements that the text has changed
if (this.PropertyChanged != null)
{
this.PropertyChanged(this, new PropertyChangedEventArgs("StatusText"));
}
}
}
}
/// <summary>
/// Execute shutdown tasks
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void MainWindow_Closing(object sender, CancelEventArgs e)
{
if (this.depthFrameReader != null)
{
// DepthFrameReader is IDisposable
this.depthFrameReader.Dispose();
this.depthFrameReader = null;
}
if (this.kinectSensor != null)
{
this.kinectSensor.Close();
this.kinectSensor = null;
}
}
/// <summary>
/// Handles the user clicking on the screenshot button
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void ScreenshotButton_Click(object sender, RoutedEventArgs e)
{
if (this.depthBitmap != null)
{
// create a png bitmap encoder which knows how to save a .png file
BitmapEncoder encoder = new PngBitmapEncoder();
// create frame from the writable bitmap and add to encoder
encoder.Frames.Add(BitmapFrame.Create(this.depthBitmap));
string time = System.DateTime.UtcNow.ToString("hh'-'mm'-'ss", CultureInfo.CurrentUICulture.DateTimeFormat);
string myPhotos = Environment.GetFolderPath(Environment.SpecialFolder.MyPictures);
string path = Path.Combine(myPhotos, "KinectScreenshot-Depth-" + time + ".png");
// write the new file to disk
try
{
// FileStream is IDisposable
using (FileStream fs = new FileStream(path, FileMode.Create))
{
encoder.Save(fs);
}
this.StatusText = string.Format(CultureInfo.CurrentCulture, Properties.Resources.SavedScreenshotStatusTextFormat, path);
}
catch (IOException)
{
this.StatusText = string.Format(CultureInfo.CurrentCulture, Properties.Resources.FailedScreenshotStatusTextFormat, path);
}
}
}
/// <summary>
/// Handles the depth frame data arriving from the sensor
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void Reader_FrameArrived(object sender, DepthFrameArrivedEventArgs e)
{
bool depthFrameProcessed = false;
using (DepthFrame depthFrame = e.FrameReference.AcquireFrame())
{
if (depthFrame != null)
{
// the fastest way to process the body index data is to directly access
// the underlying buffer
using (Microsoft.Kinect.KinectBuffer depthBuffer = depthFrame.LockImageBuffer())
{
// verify data and write the color data to the display bitmap
if (((this.depthFrameDescription.Width * this.depthFrameDescription.Height) == (depthBuffer.Size / this.depthFrameDescription.BytesPerPixel)) &&
(this.depthFrameDescription.Width == this.depthBitmap.PixelWidth) && (this.depthFrameDescription.Height == this.depthBitmap.PixelHeight))
{
// Note: In order to see the full range of depth (including the less reliable far field depth)
// we are setting maxDepth to the extreme potential depth threshold
ushort maxDepth = ushort.MaxValue;
// If you wish to filter by reliable depth distance, uncomment the following line:
//// maxDepth = depthFrame.DepthMaxReliableDistance
this.ProcessDepthFrameData(depthBuffer.UnderlyingBuffer, depthBuffer.Size, depthFrame.DepthMinReliableDistance, maxDepth);
depthFrameProcessed = true;
}
}
}
}
if (depthFrameProcessed)
{
this.RenderDepthPixels();
}
}
/// <summary>
/// Directly accesses the underlying image buffer of the DepthFrame to
/// create a displayable bitmap.
/// This function requires the /unsafe compiler option as we make use of direct
/// access to the native memory pointed to by the depthFrameData pointer.
/// </summary>
/// <param name="depthFrameData">Pointer to the DepthFrame image data</param>
/// <param name="depthFrameDataSize">Size of the DepthFrame image data</param>
/// <param name="minDepth">The minimum reliable depth value for the frame</param>
/// <param name="maxDepth">The maximum reliable depth value for the frame</param>
private unsafe void ProcessDepthFrameData(IntPtr depthFrameData, uint depthFrameDataSize, ushort minDepth, ushort maxDepth)
{
// depth frame data is a 16 bit value
ushort* frameData = (ushort*)depthFrameData;
// convert depth to a visual representation
for (int i = 0; i < (int)(depthFrameDataSize / this.depthFrameDescription.BytesPerPixel); ++i)
{
// Get the depth for this pixel
ushort depth = frameData[i];
// To convert to a byte, we're mapping the depth value to the byte range.
// Values outside the reliable depth range are mapped to 0 (black).
this.depthPixels[i] = (byte)(depth >= minDepth && depth <= maxDepth ? (depth / MapDepthToByte) : 0);
}
}
/// <summary>
/// Renders color pixels into the writeableBitmap.
/// </summary>
private void RenderDepthPixels()
{
this.depthBitmap.WritePixels(
new Int32Rect(0, 0, this.depthBitmap.PixelWidth, this.depthBitmap.PixelHeight),
this.depthPixels,
this.depthBitmap.PixelWidth,
0);
}
/// <summary>
/// Handles the event which the sensor becomes unavailable (E.g. paused, closed, unplugged).
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void Sensor_IsAvailableChanged(object sender, IsAvailableChangedEventArgs e)
{
// on failure, set the status text
this.StatusText = this.kinectSensor.IsAvailable ? Properties.Resources.RunningStatusText
: Properties.Resources.SensorNotAvailableStatusText;
}
}
}

private unsafe void ProcessDepthFrameData(IntPtr depthFrameData, uint depthFrameDataSize, ushort minDepth, ushort maxDepth)
{
// CONVERT IN TO ANY VALID FORMAT YOU WANT
// THEN HERE YOU HAVE DEPTH DATA ARRAY, SAVE IT AS YOU LIKE
ushort* frameData = (ushort*)depthFrameData;
}
If you need more efficiency, you can try something like
depthFrame.CopyFrameDataToBuffer(buffer);
In Reader_FrameArrived function.

The example provided in the Kinect SDK saves the PNG's in 8-bit format (0-255) therefore you loose the depth values. The issue is that WPF(?) cannot display 16-bit image data on screen (i believe due to GDI+ restrictions as mentioned in another SO question)
To save the depth images as Raw 16-bit PNG's; a couple of things need to be done.
WriteableBitmap item with PixelFormats.Gray16 needs to be declared, instead of the PixelFormats.Gray8
Two separate WritePixels need to be called, one for displaying the image in (byte) format and another for saving the depth image in 16-bit (ushort) format
Lastly, the depth data acquired needs to be processed twice, once in a byte array for displaying on screen, and once in ushort array for saving.
Note that the above can be modified as required (or for performance reasons if req) but for me, this works just fine and i can display the depth images and save the raw depth data aswell.
The code is as follows:
Initialize a new var:
private ushort[] raw = null;
private WriteableBitmap rawPNG = null;
In MainWindow Create two writeableBitmap's
this.depthBitmap = new WriteableBitmap(this.depthFrameDescription.Width, this.depthFrameDescription.Height, 96.0, 96.0, PixelFormats.Gray8, null);
this.rawPNG= new WriteableBitmap( this.depthFrameDescription.Width, this.depthFrameDescription.Height, 96.0, 96.0, PixelFormats.Gray16, null );
In ProcessDepthFrameData assign the values:
// FOR DISPLAYING ON SCREEN:
// To convert to a byte, we're mapping the depth value to the byte range.
// Values outside the reliable depth range are mapped to 0 (black).
this.depthPixels[i] = (byte)(depth >= minDepth && depth <= maxDepth ? (depth / MapDepthToByte) : 0);
// FOR SAVING RAW PNG 16-BIT DATA
// Keeping the raw depth values after clipping them to the max depth. The data is converted to ushort which is 16bit data
raw[i] = (ushort)(depth >= minDepth && depth <= maxDepth ? (depth) : 0);
In RenderDepthPixels, create teh two images:
// Rendering frame for displaying on the screen, in 8-Bit grayscale format
depthBitmap.WritePixels(
new Int32Rect( 0, 0, depthBitmap.PixelWidth, depthBitmap.PixelHeight ),
depthPixels,
depthBitmap.PixelWidth,
0 );
// Rendering frame for saving as raw-16-bit PNG file
rawPNG.WritePixels(
new Int32Rect( 0, 0, depthBitmap.PixelWidth, depthBitmap.PixelHeight ),
raw,
depthBitmap.PixelWidth * 2,
0 );
And lastly, in ScreenshotButton_Click, save the file:
// TO SAVE THE DISPLAYED 8-BIT PNG
//encoder.Frames.Add( BitmapFrame.Create( this.depthBitmap) );
// TO SAVE THE RAW 16-BIT PNG
encoder.Frames.Add( BitmapFrame.Create( this.rawPNG ) );
The results can be verified by loading the depth image in Matlab. Imread shows a Uint16 image format and the depth values are retained in the matrix. Or by just checking the png file properties :)
I have shared the complete code that is working for me at Pastebin. Hopefully it should work for you aswell, and solves the issue.

How do I draw the outline of a collection of rectangles?

As part of a project I'm working on I have to store and restore magic wand regions from an image. To obtain the data for storage, I'm utilizing the GetRegionData method. As the specification specifies, this method:
Returns a RegionData that represents the information that describes this Region.
I store the byte[] kept in the RegionData.Data property in a base64 string, so I can retrieve the RegionData later through a somewhat unconventional method:
// This "instantiates" a RegionData object by simply initiating an object and setting the object type pointer to the specified type.
// Constructors don't run, but in this very specific case - they don't have to. The RegionData's only member is a "data" byte array, which we set right after.
var regionData =
(RegionData)FormatterServices.GetUninitializedObject(typeof(RegionData));
regionData.Data = bytes;
I then create a Region and pass the above RegionData object in the constructor and call GetRegionScans to obtain the rectangle objects which comprise the region:
var region = new Region(regionData);
RectangleF[] rectangles = region.GetRegionScans(new Matrix());
This way I end up with a collection of rectangles that I use to draw and reconstruct the region. I have isolated the entire drawing process to a WinForms application, and I'm using the following code to draw this collection of rectangles on an image control:
using (var g = Graphics.FromImage(picBox.Image))
{
var p = new Pen(Color.Black, 1f);
var alternatePen = new Pen(Color.BlueViolet, 1f);
var b = new SolidBrush(picBox.BackColor);
var niceBrush = new SolidBrush(Color.Orange);
foreach (var r in rectangles)
{
g.DrawRectangle(p,
new Rectangle(new Point((int)r.Location.X, (int)r.Location.Y),
new Size((int)r.Width, (int)r.Height)));
}
}
The above code results in the following being rendered in my picture control:
The outline here is correct - that's exactly what I originally marked with my magic wand tool. However, as I'm drawing rectangles, I also end up with horizontal lines, which weren't a part of the original magic wand selection. Because of this, I can't view the actual image anymore, and my wand tool now makes the image useless.
I figured I'd only draw the left and right edges of each of the rectangles on the screen, so I'd end up with a bunch of points on the screen which outline the image for me. To do this, I tried the following code:
var pointPair = new[]
{
new Point((int) r.Left, (int) r.Y),
new Point((int) r.Right, (int) r.Y)
};
g.DrawRectangle(p, pointPair[0].X, pointPair[0].Y, 1, 1);
g.DrawRectangle(p, pointPair[1].X, pointPair[1].Y, 1, 1);
And the result can be observed below:
Though closer, it's still no cigar.
I'm looking for a way to connect the dots in order to create an outline of the rectangles in my region. Something that's trivial for humans to do, but I can not figure out for the life of me how to instruct a computer to do this for me.
I've already tried creating new points from each of the rectangles by calculating the most adjacent points on both the Left and Right points of each rectangle, and rendering those as individual rectangles, but to no avail.
Any help would be greatly appreciated, as I'm really at a loss here.
Thanks!
Solution
Thanks to Peter Duniho's answer, I managed to solve this problem. I'm including the SafeHandle wrapper classes and the code I've used to make this work below, for the sake of completeness.
Drawing code
The code below is what I've used to draw the region outline.
private void DrawRegionOutline(Graphics graphics, Color color, Region region)
{
var regionHandle = new SafeRegionHandle(region, region.GetHrgn(graphics));
var deviceContext = new SafeDeviceContextHandle(graphics);
var brushHandle = new SafeBrushHandle(color);
using (regionHandle)
using (deviceContext)
using (brushHandle)
FrameRgn(deviceContext.DangerousGetHandle(), regionHandle.DangerousGetHandle(), brushHandle.DangerousGetHandle(), 1, 1);
}
SafeHandleNative
Small wrapper around SafeHandleZeroOrMinusOneIsInvalid to ensure cleanup after we're done with the handles.
[HostProtection(MayLeakOnAbort = true)]
[SuppressUnmanagedCodeSecurity]
public abstract class SafeNativeHandle : SafeHandleZeroOrMinusOneIsInvalid
{
#region Platform Invoke
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
protected internal static extern bool CloseHandle(IntPtr hObject);
#endregion
/// <summary>
/// Initializes a new instance of the <see cref="SafeNativeHandle"/> class.
/// </summary>
protected SafeNativeHandle() : base(true)
{}
/// <summary>
/// Initializes a new instance of the <see cref="SafeNativeHandle"/> class.
/// </summary>
/// <param name="handle">The handle.</param>
protected SafeNativeHandle(IntPtr handle)
: base(true)
{
SetHandle(handle);
}
}
I've created three other wrappers, one for the Region object, one for the Brush and one for the device context handles. These all inherit from SafeNativeHandle, but in order not to spam I'll only provide the one I've used for the region below. The other two wrappers are virtually identical, but use the respective Win32 API required to clean up their own resources.
public class SafeRegionHandle : SafeNativeHandle
{
private readonly Region _region;
/// <summary>
/// Initializes a new instance of the <see cref="SafeRegionHandle" /> class.
/// </summary>
/// <param name="region">The region.</param>
/// <param name="handle">The handle.</param>
public SafeRegionHandle(Region region, IntPtr handle)
{
_region = region;
base.handle = handle;
}
/// <summary>
/// When overridden in a derived class, executes the code required to free the handle.
/// </summary>
/// <returns>
/// true if the handle is released successfully; otherwise, in the event of a catastrophic failure, false. In this case, it generates a releaseHandleFailed MDA Managed Debugging Assistant.
/// </returns>
[ReliabilityContract(Consistency.WillNotCorruptState, Cer.MayFail)]
protected override bool ReleaseHandle()
{
try
{
_region.ReleaseHrgn(handle);
}
catch
{
return false;
}
return true;
}
}

I'm still not entirely sure I understand the question. However, it sounds to me as though you simply want to draw the given region by outlining it, rather than filling it.
Unfortunately, as far as I know the .NET API does not support this. However, the native Windows API does. Here is some code that should do what you want:
[DllImport("gdi32")]
static extern bool FrameRgn(System.IntPtr hDC, System.IntPtr hRgn, IntPtr hBrush, int nWidth, int nHeight);
[DllImport("gdi32")]
static extern IntPtr CreateSolidBrush(uint colorref);
[DllImport("gdi32.dll")]
static extern bool DeleteObject([In] IntPtr hObject);
[StructLayout(LayoutKind.Explicit)]
struct COLORREF
{
[FieldOffset(0)]
public uint colorref;
[FieldOffset(0)]
public byte red;
[FieldOffset(1)]
public byte green;
[FieldOffset(2)]
public byte blue;
public COLORREF(Color color)
: this()
{
red = color.R;
green = color.G;
blue = color.B;
}
}
void DrawRegion(Graphics graphics, Color color, Region region)
{
COLORREF colorref = new COLORREF(color);
IntPtr hdc = IntPtr.Zero, hbrush = IntPtr.Zero, hrgn = IntPtr.Zero;
try
{
hrgn = region.GetHrgn(graphics);
hdc = graphics.GetHdc();
hbrush = CreateSolidBrush(colorref.colorref);
FrameRgn(hdc, hrgn, hbrush, 1, 1);
}
finally
{
if (hrgn != IntPtr.Zero)
{
region.ReleaseHrgn(hrgn);
}
if (hbrush != IntPtr.Zero)
{
DeleteObject(hbrush);
}
if (hdc != IntPtr.Zero)
{
graphics.ReleaseHdc(hdc);
}
}
}
Call the DrawRegion() method from your Paint event handler or other appropriate context where you have a Graphics instance, such as drawing into an Image object as in your example.
Obviously you could make this an extension method for more convenience. Also, while in this example I am dealing with the initialization and releasing of the handles directly, a better implementation would wrap the handles in appropriate SafeHandle subclasses, so that you can conveniently use using instead of try/finally, and to get the backup of finalization (in case you forget to dispose).

Using a method to get Embedded Font Causes Protected Memory Error

I am using this code to get an embedded font:
/// <summary>
/// Returns an Embedded Font
/// </summary>
/// <param name="ImagePath">String begins with namespace e.g MyProgram.Image.png</param>
/// <returns></returns>
public static Font GetEmbeddedFont(string FontPath, float Size)
{
Font _font = null;
Thread getFontThread = new Thread(() => GetFont(FontPath, Size, out _font));
getFontThread.Start();
getFontThread.Join();
return _font;
}
#region GetFont
private static void GetFont(string FontPath, float Size, out Font FontOut)
{
Font fnt = null;
Assembly asm = Assembly.GetExecutingAssembly();
Stream resStream = asm.GetManifestResourceStream(FontPath);
if (null != resStream)
{
//
// GDI+ wants a pointer to memory, GDI wants the memory.
// We will make them both happy.
//
// First read the font into a buffer
byte[] rgbyt = new Byte[resStream.Length];
resStream.Read(rgbyt, 0, rgbyt.Length);
resStream.Close();
// Then do the unmanaged font (Windows 2000 and later)
// The reason this works is that GDI+ will create a font object for
// controls like the RichTextBox and this call will make sure that GDI
// recognizes the font name, later.
uint cFonts;
AddFontMemResourceEx(rgbyt, rgbyt.Length, IntPtr.Zero, out cFonts);
// Now do the managed font
IntPtr pbyt = Marshal.AllocCoTaskMem(rgbyt.Length);
if (null != pbyt)
{
Marshal.Copy(rgbyt, 0, pbyt, rgbyt.Length);
m_pfc = new PrivateFontCollection();
m_pfc.AddMemoryFont(pbyt, rgbyt.Length);
Marshal.FreeCoTaskMem(pbyt);
}
}
if (m_pfc.Families.Length > 0)
{
// Handy how one of the Font constructors takes a
// FontFamily object, huh? :-)
fnt = new Font(m_pfc.Families[0], Size);
}
m_pfc.Dispose();
FontOut = fnt;
}
I use a Thread to attempt to wait for it to finish, because the error seems to occur, if this method is called multiple times within a short space of time.
How can I stop this error occuring, I think it has something to do with the method being called within quick succession of each other.
I get the exception here:
_font = value;
using (Graphics g = _parent.CreateGraphics())
{
SizeF soize = g.MeasureString(_text, _font);
_size = new Size((int)soize.Width, (int)soize.Height);
_width = _size.Width;
_height = _size.Height;
}
On the line g.MeasureString(_text, _font);
However I know that the error is in the GetEmbeddedFont Method, as it only throws an error if the font is set using the GetEmbeddedFont method.
It will work fine once, but if it is used a second time to shortly after the first, it will throw the error.
And if I debug the code, _font returns this:
{Name = '((System.Drawing.Font)(_font)).fontFamily.Name' threw an exception of type 'System.ArgumentException' Size=15.0}

There's a documentation problem with AddMemoryFont(), it doesn't specify how long the pointer needs to remain valid. I've always chosen the conservative route and made sure to not release the memory until after the program stops using the private font. That has worked well, no access violations.
I thus strongly recommend you remove the Marshal.FreeCoTaskMem() call. Either entirely, Windows automatically cleans up when the program terminates, or move it to, say, the FormClosed event. The same goes for your m_pfc.Dispose() call. Don't release resources until you're done with them.

.NET and Direct2D rapid draw crash

General purpose
I am developing a .NET user control that can be used for multipurpose graphics rendering using Direct2D. This control is based on the ID2D1HwndRenderTarget interface with a ID2D1BitmapRenderTarget back-buffer/double-buffer.
Background
I have written a wrapper using C++/CLI that exposes a set of managed wrapper objects for Direct2D, and I have a designed a Singleton resource manager for Direct2D bitmaps that code can send to the binary data to, and get a resource key back from, which code can then pass in a message to the user control to render a bitmap.
Usage so far
I have been able to load bitmaps, JPEGs, custom imaging formats and the like and send them to the Direct2D render control all with no issue. I've written a custom video decoder that could load frames from a 15-fps video (decoding in a background thread) and render them from events raised by a Win32 multimedia timer, no problem.
Issue
My issue is that when I attempted to expand out from a single multimedia format into something more flexible (specifically connecting to LibAV/ffmpeg), the Direct2D render control starts to crash display drivers. This does not happen when frames are rendered sequentially (using a button to render the next frame, rather than a timer). It also does not happen straightaway. It also does not happen when I block inside the Win32 timer callback, but does when I use a Mutex to raise a thread that will free the timer callback up and let the interim thread instruct the control to render.
Symptoms
If I start a timer and attach to its elapsed event methods that will cause the next render, it will typically play fine for around 2-20 seconds. Using smaller video, I can get longer playback before the issue starts. If I play back 1080p video, I can get the crash usually within 5 seconds without contest. Playback will start fine, maybe have a hiccup here or there before catching up.
Eventually, I will start to see flickering, as if the background color is rendered but the frame is not. Additional frames may or may not later render, but this is very short-lived.
After the flicker, if I stop the timer quickly enough there will be no crash.
If I do not stop in time:
If I'm lucky (maybe 10% of the time), frames will stop rendering, be stuck on one frame until I resize the control, at which point only black is drawn to the entire control. (I've read that this may indicate a lost device context for DirectX rendering, but have not seen much more.)
If I'm unlucky, the display driver will crash, Windows recovers and restarts the driver, after which point the EndDraw will finally tell me that an error has occurred and return D2DERR_RECREATE_TARGET.
If I am really unlucky, the display will start to look like a kaleidoscope and I'll have to power off my machine, and I just wasted 5 minutes booting, logging on, loading Visual Studio and my solution, loading the video and lost all my debugging data.
I want to think that there is some sort of race condition that I am missing, but every time I run through the rendering code, it appears that it should be properly locked.
Render Control's Code
using System;
using System.Drawing;
using System.Windows.Forms;
using Direct2D = Bardez.Projects.DirectX.Direct2D;
using ExternalPixelEnums = Bardez.Projects.FileFormats.MediaBase.Video.Pixels.Enums;
using Bardez.Projects.DirectX.Direct2D;
using Bardez.Projects.FileFormats.MediaBase.Video;
using Bardez.Projects.FileFormats.MediaBase.Video.Enums;
using Bardez.Projects.Win32;
namespace Bardez.Projects.Output.Visual
{
/// <summary>Represents a rendering target for Direct2D.</summary>
/// <remarks>
/// To use this control, an external component will need to supply bitmap data. This does not need to be a GDI+ Bitmap class.
/// However, the container for this control will need to push data into this control.
/// So, in the case of a movie player, we'd see the following model:
/// * decompress a frame.
/// * push frame to control
/// * invoke Invalidate
/// * control will render the bitmap
/// * sleep just a little bit
/// * go back to first step
/// </remarks>
public class Direct2dRenderControl : VisualRenderControl
{
/*
* Locking orientation:
* There are two rendering targets: a GDI display and a bitmap back buffer.
* There are 5 'real' locking operations:
* Rendering to the GDI display (OnPaint)
* Rendering to the back buffer (DrawBitmapToBuffer, DiscardCurrentBuffer)
* Setting the current displayed frame (SetRenderFrame)
* Resource Freeing (FreeFrameResource)
* Resizing (OnResize)
*
* Briefly, the overarching effects of these five are:
* Rendering
* Utilizes the buffer and the display
* Back Buffer
* Utilizes the buffer and if mid-render could affect this display
* Set Frame
* Sets the buffer's displayed image
* Resource Freeing
* Affects the buffer if a resource is refernced
* Resizing
* Resizes the render control and the bitmap, uses the frame set by set
*
* Locking plan:
* Resize should block set, free and back buffer
* Render should block back buffer, control
* Set frame should block resize and back buffer
* Free should block back buffer,
* Back buffer should block rendering, setting, resizing, freeing
*
* Basically, lock everything at the process level, and not at the access level.
*/
#region Fields
/// <summary>Represents a Win32 HWND render target for Direct2D</summary>
private ControlRenderTarget ctrlRenderTarget;
/// <summary>Represents a drawing buffer</summary>
private BitmapRenderTarget bmpRenderTarget;
/// <summary>Represents a drawing buffer, used solely to create bitmaps</summary>
private BitmapRenderTarget resourceBmpRenderTarget;
/// <summary>Represents a buffer drawing command lock</summary>
private Object controlBufferLock;
/// <summary>Represents a paint/render drawing command lock</summary>
private Object controlPaintRenderLock;
/// <summary>Locking object reference for resource management (memory bitmaps, etc.)</summary>
private Object resourceLock;
/// <summary>Represents the key to accessing the currently set key</summary>
protected Int32 currentFrameKey;
#endregion
#region Properties
/// <summary>Indicates whether there is a frame set for this control</summary>
protected Boolean HasFrameSet
{
get { return currentFrameKey > -1; }
}
/// <summary>Exposes a wrapper for the bitmap render target</summary>
protected BitmapRenderTarget BmpRenderTarget
{
get { return this.bmpRenderTarget; }
set
{
lock (this.controlBufferLock)
{
if (this.bmpRenderTarget != null)
this.bmpRenderTarget.Dispose();
this.bmpRenderTarget = value;
}
}
}
/// <summary>Exposes a wrapper for the bitmap render target</summary>
protected BitmapRenderTarget ResourceBmpRenderTarget
{
get { return this.resourceBmpRenderTarget; }
set
{
lock (this.resourceLock)
{
if (this.resourceBmpRenderTarget != null)
this.resourceBmpRenderTarget.Dispose();
this.resourceBmpRenderTarget = value;
}
}
}
/// <summary>Represents a Win32 HWND render target for Direct2D</summary>
protected ControlRenderTarget CtrlRenderTarget
{
get { return this.ctrlRenderTarget; }
set
{
lock (this.controlPaintRenderLock)
{
if (this.ctrlRenderTarget != null)
this.ctrlRenderTarget.Dispose();
this.ctrlRenderTarget = value;
}
}
}
#endregion
#region Construction
/// <summary>Default constructor</summary>
public Direct2dRenderControl() : base()
{
this.controlBufferLock = new Object();
this.controlPaintRenderLock = new Object();
this.resourceLock = new Object();
this.currentFrameKey = -1;
this.InitializeControlDirect2D();
}
/// <summary>Initializes the Direct2D</summary>
protected void InitializeControlDirect2D()
{
//disable Windows background draw
this.SetStyle(ControlStyles.UserPaint | ControlStyles.AllPaintingInWmPaint, true);
// Build options on the Control render target
PixelFormat format = new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.Unknown); //32-bit color, pure alpha
DpiResolution res = Direct2dResourceManager.Instance.Factory.GetDesktopDpi();
RenderTargetProperties rtProp = new RenderTargetProperties(RenderTargetType.Default, format, res, RenderTargetUsage.GdiCompatible, DirectXVersion.DirectX9);
//Build out control render target properties
HwndRenderTargetProperties hwndProp = new HwndRenderTargetProperties(this.Handle, new SizeU(this.Size), PresentationOptions.RetainContents);
lock (this.controlPaintRenderLock)
{
// populate the Control rendering target
ResultCode result = Direct2dResourceManager.Instance.Factory.CreateHwndRenderTarget(rtProp, hwndProp, out this.ctrlRenderTarget);
lock (this.controlBufferLock)
{
// create a bitmap rendering targets
this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.bmpRenderTarget);
lock (this.resourceLock)
this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.resourceBmpRenderTarget);
}
}
}
#endregion
#region Destruction
/// <summary>Disposal code; releases unmanaged resources</summary>
/// <param name="disposing">True indicates to dispose managed resources</param>
protected override void Dispose(Boolean disposing)
{
this.ResourceBmpRenderTarget = null; //property disposes
this.BmpRenderTarget = null; //property disposes
this.CtrlRenderTarget = null; //property disposes
base.Dispose(disposing);
}
/// <summary>Disposal</summary>
~Direct2dRenderControl()
{
this.Dispose();
}
#endregion
#region Event Raising
/// <summary>Draws the output, then raises the paint event</summary>
/// <param name="e">Painting Event arguments</param>
protected override void OnPaint(PaintEventArgs e)
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
this.SuspendLayout();
this.OnPaintBackground(e);
Direct2D.Bitmap bmp;
ResultCode result;
result = this.BmpRenderTarget.GetBitmap(out bmp);
Direct2D.RectangleF rect = new Direct2D.RectangleF(e.ClipRectangle);
this.CtrlRenderTarget.BeginDraw();
this.CtrlRenderTarget.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);
result = this.CtrlRenderTarget.EndDraw();
bmp.Dispose();
if (result != ResultCode.Success_OK)
throw new ApplicationException(String.Format("Error encountered during draw: '{0}'", result.ToString()));
base.OnPaint(e);
this.ResumeLayout();
}
}
}
/// <summary>Overides the resize method</summary>
/// <param name="e">Parameters for the resize event</param>
protected override void OnResize(EventArgs e)
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
//Null check since resize fires before it is constructed, with the size event during parent's InitializeComponent
if (this.CtrlRenderTarget != null)
{
this.SuspendLayout();
//resize the existing control rendering target
this.CtrlRenderTarget.Resize(new SizeU(this.Size));
//I also need to resize the buffer, but can't. Instead, create a new one, then copy the existing one. Kind of lame.
this.ResizeBitmapRenderTarget();
base.OnResize(e);
//cause another draw
this.Invalidate(new Rectangle(new Point(0, 0), this.Size));
this.ResumeLayout();
}
}
}
}
/// <summary>Overridden Paint background method</summary>
/// <param name="e">Paint event arguments</param>
/// <remarks>
/// Made empty to avoid GDI and Direct2D writing to the same control; when moving the control around
/// or rendering at high speeds, the dreaded WinForms flicker was introduced.
/// The background painting can be found in the <see cref="FillBufferRenderTarget"/> method,
/// which fills the bitmap back buffer with the control's background color.
/// </remarks>
protected override void OnPaintBackground(PaintEventArgs e) { }
#endregion
#region Drawing
/// <summary>Resizes the bitmap rendering target buffer</summary>
/// <remarks>Does not lock the GDI render target. The OnResize or other callers lock instead.</remarks>
protected void ResizeBitmapRenderTarget()
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
using (BitmapRenderTarget bmpCurr = this.BmpRenderTarget)
{
BitmapRenderTarget bmpNew;
ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);
this.DuplicateDoubleBufferContents(bmpNew);
//Property disposes and locks
this.BmpRenderTarget = bmpNew;
}
}
}
}
/// <summary>Draws a Bitmap to the render buffer</summary>
/// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp, Point2dF origin)
{
lock (this.controlBufferLock)
{
lock (this.resourceLock)
{
Direct2D.SizeF bmpSize = bmp.GetSize();
Single width = bmpSize.Width > this.BmpRenderTarget.Size.Width ? this.BmpRenderTarget.Size.Width : bmpSize.Width;
Single height = bmpSize.Height > this.BmpRenderTarget.Size.Height ? this.BmpRenderTarget.Size.Height : bmpSize.Height;
Direct2D.RectangleF destRect = new Direct2D.RectangleF(origin.X, origin.X + width, origin.Y, origin.Y + height);
Direct2D.RectangleF srcRect = new Direct2D.RectangleF(0.0F, width, 0.0F, height);
this.BmpRenderTarget.BeginDraw();
this.FillBufferRenderTarget(this.BmpRenderTarget);
// do the actual draw
this.BmpRenderTarget.DrawBitmap(bmp, destRect, 1.0F, BitmapInterpolationMode.Linear, srcRect);
//tell Direct2D that a paint operation is ending
ResultCode result = this.BmpRenderTarget.EndDraw();
}
}
}
/// <summary>Draws a Bitmap to the render buffer</summary>
/// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp)
{
this.DrawBitmapToBuffer(bmp, new Point2dF(0.0F, 0.0F));
}
/// <summary>Duplicates the bitmap behind the existing rendering target, and drawing it to a new one, discarding the current and setting the new.</summary>
/// <remarks>Does not lock any references, as the outside method locks</remarks>
protected void DuplicateDoubleBufferContents(BitmapRenderTarget bmpNew)
{
Direct2D.Bitmap bmp = null;
ResultCode result = ResultCode.Success_OK;
if (this.HasFrameSet)
bmp = Direct2dResourceManager.Instance.GetBitmapResource(this.currentFrameKey);
else
result = this.BmpRenderTarget.GetBitmap(out bmp);
bmpNew.BeginDraw();
this.FillBufferRenderTarget(bmpNew);
//calculate the size to copy
Direct2D.SizeF bmpSize = bmp.GetSize();
Single width = bmpSize.Width > this.CtrlRenderTarget.Size.Width ? this.CtrlRenderTarget.Size.Width : bmpSize.Width;
Single height = bmpSize.Height > this.CtrlRenderTarget.Size.Height ? this.CtrlRenderTarget.Size.Height : bmpSize.Height;
//Determine the copy rectangle
Direct2D.RectangleF rect = new Direct2D.RectangleF(0, width, 0, height);
//Copy
bmpNew.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);
//conditionally disose the bitmap, don't if it is in the manager
if (!this.HasFrameSet)
bmp.Dispose();
result = bmpNew.EndDraw();
}
/// <summary>Discards the current buffer and replaces it with a blank one.</summary>
protected void DiscardCurrentBuffer()
{
lock (this.controlBufferLock)
{
BitmapRenderTarget bmpNew;
// create a bitmap rendering target
ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);
bmpNew.BeginDraw();
this.FillBufferRenderTarget(bmpNew);
result = bmpNew.EndDraw();
//property locks, so no lock here
this.BmpRenderTarget = bmpNew; //replace the old buffer
}
}
/// <summary>Fills the buffer render target with the background color</summary>
/// <param name="renderTarget">Bitmap render target to fill</param>
protected void FillBufferRenderTarget(BitmapRenderTarget renderTarget)
{
SolidColorBrush brush = null;
ResultCode result = renderTarget.CreateSolidColorBrush(new ColorF(this.BackColor), out brush);
renderTarget.FillRectangle(new Direct2D.RectangleF(new Rectangle(new Point(0, 0), this.Size)), brush);
brush.Dispose();
}
#endregion
#region Frame Resources
/// <summary>Posts a Frame resource to the resource manager and returns a unique key to access it.</summary>
/// <param name="resource">Frame to be posted.</param>
/// <returns>A unique Int32 key</returns>
public override Int32 AddFrameResource(Frame resource)
{
lock (this.resourceLock)
{
//create the bitmap
BitmapProperties properties = new BitmapProperties(new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.PreMultiplied), Direct2dResourceManager.Instance.Factory.GetDesktopDpi());
SizeU dimensions = new SizeU(Convert.ToUInt32(resource.Pixels.Metadata.Width), Convert.ToUInt32(resource.Pixels.Metadata.Height));
Direct2D.Bitmap bmp = null;
ResultCode result = this.ResourceBmpRenderTarget.CreateBitmap(dimensions, properties, out bmp);
Byte[] data = resource.Pixels.GetPixelData(ExternalPixelEnums.PixelFormat.RGBA_B8G8R8A8, ScanLineOrder.TopDown, 0, 0);
result = bmp.CopyFromMemory(new RectangleU(dimensions), data, Convert.ToUInt32(resource.Pixels.Metadata.Width * 4));
return Direct2dResourceManager.Instance.AddFrameResource(bmp);
}
}
/// <summary>Frees a Bitmap resource in the resource manager and Disposes of it.</summary>
/// <param name="frameKey">Direct2D Bitmap key to be Disposed.</param>
public override void FreeFrameResource(Int32 frameKey)
{
lock (this.resourceLock)
{
if (frameKey > -1)
Direct2dResourceManager.Instance.FreeFrameResource(frameKey);
}
}
#endregion
#region Frame Setting
/// <summary>Sets the frame to be rendered to the User Control</summary>
/// <param name="key">Frame key to set as current image</param>
public override void SetRenderFrame(Int32 key)
{
this.SetRenderFrame(key, 0, 0);
}
/// <summary>Sets the frame to be rendered to the User Control</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
public override void SetRenderFrame(Int32 key, Int64 originX, Int64 originY)
{
lock (this.controlBufferLock)
{
if (key > -1)
this.DrawBitmapToBuffer(Direct2dResourceManager.Instance.GetBitmapResource(key), new Point2dF(Convert.ToSingle(originX), Convert.ToSingle(originY)));
else
this.DiscardCurrentBuffer();
this.currentFrameKey = key;
}
}
/// <summary>This method invokes the rendering. For use by the appliation to tell the control to change images on demand.</summary>
public void Render()
{
this.Invalidate();
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
public virtual void SetRenderFrameAndRender(Int32 key)
{
this.SetRenderFrameAndRender(key, false);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="freePreviousFrame">
/// Flag indicating whether to dispose of the previous image set
/// (in the case of transient images, such as composite images for a game or high-frame video playback)
/// </param>
public virtual void SetRenderFrameAndRender(Int32 key, Boolean freePreviousFrame)
{
this.SetRenderFrameAndRender(key, 0, 0, freePreviousFrame);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY)
{
this.SetRenderFrameAndRender(key, originX, originY, false);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
/// <param name="freePreviousFrame">
/// Flag indicating whether to dispose of the previous image set
/// </param>
public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY, Boolean freePreviousFrame)
{
lock (this.controlBufferLock)
{
Int32 previousFrameKey = this.currentFrameKey;
this.SetRenderFrame(key, originX, originY);
this.Render();
if (freePreviousFrame)
this.FreeFrameResource(previousFrameKey);
}
}
#endregion
}
}
Win32 Multimedia Timer Callback's Code
Excluded for size, but here is the relevant code, linking to "winmm.dll", EntryPoint = "timeSetEvent"
/// <summary>Callback method for WIn32 API</summary>
protected virtual void Win32Callback(UInt32 timerId, UInt32 message, IntPtr user, IntPtr param1, IntPtr param2)
{
TimeSpan raise;
lock (this.timerLock)
{
//get system time for start time
UInt32 uptime = Win32MultimediaTimeFunctions.GetEnvironmentUptime();
Int32 upTimeSec = (Int32)(uptime / 1000);
Int32 upTimeMilliSec = (Int32)(uptime % 1000);
TimeSpan WinUpTime = new TimeSpan(0, 0, 0, upTimeSec, upTimeMilliSec);
raise = WinUpTime - this.StartTime;
//this.elapsed(raise);
this.RaiseTimer(raise);
}
//Note: outside the lock, this.elapse(raise) kills the display driver
//inside, without threading, it is fine.
//inside, with threading, the display driver will crash after a bit of playback
}
protected void RaiseTimer(TimeSpan elapsedTime)
{
Thread timerEvent = new Thread(() => { this.RunThread(elapsedTime); });
timerEvent.IsBackground = true;
timerEvent.Name = "Timer callback";
timerEvent.Start();
}
/// <summary>Raises the timer on a separate thread</summary>
protected void RunThread(TimeSpan elapsedTime)
{
if (this.threadLock.WaitOne(0))
{
this.elapsed(elapsedTime);
this.threadLock.ReleaseMutex();
}
}
Conclusion
After 3 weeks, I'm not ashamed to say I'm baffled by this behavior, and looking for any sort of input for identifying a race condition, something that I might be missing, architecturally speaking, about Direc2D or where I'm just showing my ignorance.

All locks statements in the code seems to be an over complicated solution. Also creating COM objects (like Direct2D1) from constructors and using them from thread UI methods is usually not a good practice.
I would simplify your application by performing all device/objects creation and rendering on the same thread, also by keeping your rendering code outside the form. Ideally, this should be totally independent, as you could render to any kind of windows/controls via the HWND or to a DXGI surface, or to a Windows 8 Metro surface.
Also, as Aren suggest, instead of using your own Direct2D1 wrapper, you should rely on an existing robust wrapper like SharpDX. It will save you to maintain a costly layer and help you to focus on your specific application.