We Keep Coding

C# passing property of viewmodel into dispatcher

I have a method which updates the contents of a WriteableBitmap (FrameDataNew) which is a property in my viewmodel (VM):
public WriteableBitmap FrameDataNew
{
get { return frameDataNew; }
set
{
frameDataNew = value;
OnProptertyChanged("FrameDataNew");
}
}
private WriteableBitmap frameDataNew = null;
When I receive a new Bitmap from a gstreamer class I have written I update FrameDataNew in order to show the latest frame onscreen. the window is a simple Image control which has its source bound to FrameDataNew.
The following code works fine to do this in my event handler:
/// <summary>
/// BitmapCaptured event handler for when a new video frame is received from the IP source
/// </summary>
/// <param name="sender">The originating source of the event</param>
/// <param name="NewBitmap">The new frame in Bitmap form</param>
private void PipeLine_BitmapCaptured(object sender, Bitmap NewBitmap)
{
// render to the screen
Dispatcher.Invoke(() =>
{
// check the existence of the WriteableBitmap and also the dimensions, create a new one if required
if ((VM.FrameDataNew == null) || (VM.FrameDataNew.Width != NewBitmap.Width) || (VM.FrameDataNew.Height != NewBitmap.Height))
VM.FrameDataNew = new WriteableBitmap(NewBitmap.Width, NewBitmap.Height, NewBitmap.HorizontalResolution, NewBitmap.VerticalResolution, PixelFormats.Bgr24, null);
// lock the bitmap data so we can use it
BitmapData data = NewBitmap.LockBits(new Rectangle(0, 0, NewBitmap.Width, NewBitmap.Height), ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
// lock the backbuffer of the WritebaleBitmap so we can modify it
VM.FrameDataNew.Lock();
// Copy the bitmap's data directly to the on-screen buffers
CopyMemory(VM.FrameDataNew.BackBuffer, data.Scan0, (data.Stride * data.Height));
// Moves the back buffer to the front.
VM.FrameDataNew.AddDirtyRect(new Int32Rect(0, 0, data.Width, data.Height));
// unlock the back buffer again
VM.FrameDataNew.Unlock();
//
//NewBitmap.UnlockBits(data);
});
}
[DllImport("Kernel32.dll", EntryPoint = "RtlMoveMemory")]
public static extern void CopyMemory(IntPtr Destination, IntPtr Source, int Length);
Now I want to update my program to handle multiple pipelines and WriteableBitmaps so I can show several video feeds. The first thing I did was create a static utilities class so I can pass in the new bitmap (NewBitmap) and the WriteableBitmap (VM.FrameDataNew) I wish to update. This I then called as required when the new frame arrives:
The utilities class (just the code concerned with my question):
public static class Utils
{
/// <summary>
/// Inject the source Bitmap into the Destination WriteableBitmap
/// </summary>
/// <param name="Src">The source Bitmap</param>
/// <param name="Dest">The destination WriteableBitmap</param>
public static void InjectBitmap(Bitmap Src, WriteableBitmap Dest)
{
if ((Dest == null) || (Dest.Width != Src.Width) || (Dest.Height != Src.Height))
Dest = new WriteableBitmap(Src.Width, Src.Height, Src.HorizontalResolution, Src.VerticalResolution, PixelFormats.Bgr24, null);
BitmapData data = Src.LockBits(new Rectangle(0, 0, Src.Width, Src.Height), ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
Dest.Lock();
// Copy the bitmap's data directly to the on-screen buffers
CopyMemory(Dest.BackBuffer, data.Scan0, (data.Stride * data.Height));
// Moves the back buffer to the front.
Dest.AddDirtyRect(new Int32Rect(0, 0, data.Width, data.Height));
Dest.Unlock();
//
Src.UnlockBits(data);
}
[DllImport("Kernel32.dll", EntryPoint = "RtlMoveMemory")]
public static extern void CopyMemory(IntPtr Destination, IntPtr Source, int Length);
}
...and the way I call it:
/// <summary>
/// BitmapCaptured event handler for when a new video frame is received from the IP source
/// </summary>
/// <param name="sender">The originating source of the event</param>
/// <param name="NewBitmap">The new frame in Bitmap form</param>
private void PipeLine_BitmapCaptured(object sender, Bitmap NewBitmap)
{
// render to the screen
Dispatcher.Invoke(() =>
{
Utils.InjectBitmap(NewBitmap, VM.FrameDataNew);
});
}
The image no longer appears onscreen. Stepping through the code it looks like every time InjectBitmap() is called the destination WriteableBitmap is null? The code creates a new one in the first if statement but VM.FrameDataNew remains null?
I'm definitely at the edge of my experience with this so any help would be much appreciated.
Edit: The aim is to have an observable collection of WriteableBitmaps so I can handle several of these efficiently.

The reason is simple: you don't assign the newly created object to the FrameDataNew property, so its value remains null.
Don't forget that in C# the reference type instances are passed by reference.
So in your method you're doing the following:
void InjectBitmap(Bitmap Src, WriteableBitmap Dest)
{
if (Dest == null)
Dest = new WriteableBitmap(/* ... */);
}
But Dest is just a local variable inside of your method - the method's arguments can be seen as method's local variables.
So you assign a newly created instance to a local variable that will of course be lost when the method returns.
What you need here is a ref parameter:
void InjectBitmap(Bitmap src, ref WriteableBitmap dest)
{
if (dest == null)
dest = new WriteableBitmap(/* ... */);
}
Note that I changed the parameter name casing to match the C# style guide.
However, this won't work for properties, so InjectBitmap(NewBitmap, VM.FrameDataNew) won't compile if FrameDataNew is a property.
You could make the FrameDataNew to a field, but having a non-private mutable field is a bad idea.
You could use a temporary local variable, but it's somehow ugly:
var copy = VM.FrameDataNew;
InjectBitmap(NewBitmap, ref copy);
if (copy != VM.FrameDataNew)
VM.FrameDataNew = copy;
I would rethink your design so that you don't need all that stuff.
By the way, having multiple calls to Dispatcher.Invoke in a multithreaded environment (especially if the invoked methods are slow, and yours are) will drop your app's performance and make the UI unresponsive.

Printing Arabic string correctly in multiple lines

Currently I need to process Arabic string (اسمي مصير الطفل. من أي بلد أنت) in my project. If the string is displayed in a single line, it is displayed correctly.
However, when the font size is much bigger, the display (I am using label) is wrong.
The string is printed starting from the second line. I found that on the .Net framework we could use drawstring with StringFormatFlags.DirectionRightToLeft. However, that is not available in compact framework. So, how could i print Arabic string in multiple lines? Any advice is appreciated, thanks.

I am sorry, the following is incorrect:
Since WinCE 5 and CompactFramework v 2.0 controls like the Textbox support a RicghtToLeft property (see also http://www.danielmoth.com/Blog/rtl-problem.aspx). So you should ensure you are using CF>=2.0 and WinCE 5 base sdk (ie Windows Mobile 6.x).
Looking at the help for Textbox class, the RightToLeft is marked as NOT AVAILABLE for Compact Framework.
So, you need to write your own DrawText class that splits the words and positions them from right to left.
The native DrawText API supports the uFormat flag DT_RTLREADING (according to the online help for Windows CE 5 Platform Builder):
DT_RTLREADING
Layout in right-to-left reading order for bi-directional text when the font selected into the hdc is a Hebrew or Arabic font. The default reading order for all text is left-to-right.
There is also an option DT_WORDBREAK which I would choose for multiline text and a large enough drawing rectangle.
BUT, that gives the following result using two rectangles and two font sizes to force a wordbreak:
As I can not read that I am not sure, but I assume the wordbreak flag does not work correctly. I assume the second line in the upper part has to start from right too.
Native Code for the above:
...
#define ARABIC_TEXT L"اسمي مصير الطفل. من أي بلد أنت"
#define MulDiv(a,b,c) (((a)*(b))/(c))
...
LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
int wmId, wmEvent;
PAINTSTRUCT ps;
HDC hdc;
RECT rect;
LOGFONT lf;
HFONT hFontNew, hFontOld;
...
case WM_PAINT:
hdc = BeginPaint(hWnd, &ps);
// TODO: Add any drawing code here...
// Clear out the lf structure to use when creating the font.
memset(&lf, 0, sizeof(LOGFONT));
wsprintf(lf.lfFaceName,L"Arial Unicode MS");
GetClientRect(hWnd, &rect);
hFontNew = CreateFontIndirect(&lf);
hFontOld = (HFONT) SelectObject(hdc, hFontNew);
rect.bottom=rect.bottom/2;
lf.lfHeight=-MulDiv(16, GetDeviceCaps(hdc, LOGPIXELSY), 72);
if(DrawText(hdc, ARABIC_TEXT, -1, &rect, DT_RTLREADING | DT_WORDBREAK)==0){
DEBUGMSG(1, (L"DrawText failed with %i\n", GetLastError()));
}
GetClientRect(hWnd, &rect);
lf.lfHeight=-MulDiv(10, GetDeviceCaps(hdc, LOGPIXELSY), 72);
hFontNew = CreateFontIndirect(&lf);
hFontOld = (HFONT) SelectObject(hdc, hFontNew);
rect.top=rect.bottom/2;
if(DrawText(hdc, ARABIC_TEXT, -1, &rect, DT_RTLREADING | DT_WORDBREAK)==0){
DEBUGMSG(1, (L"DrawText failed with %i\n", GetLastError()));
}
EndPaint(hWnd, &ps);
SelectObject(hdc, hFontOld);
DeleteObject(hFontNew);
break;

The following form is an example of calling DrawText (see josef's answer) from C#:
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
protected override void OnPaint(PaintEventArgs e)
{
base.OnPaint(e);
Menu = null;
DrawText(e.Graphics, "اسمي مصير الطفل. من أي بلد أنت", Font, ClientRectangle);
}
private void DrawText(Graphics g, string text, Font font, Rectangle rectangle)
{
IntPtr dc = g.GetHdc();
RECT rect = (RECT)rectangle;
IntPtr hFont = IntPtr.Zero;
IntPtr previousFont = IntPtr.Zero;
try
{
hFont = font.ToHfont();
previousFont = SelectObject(dc, hFont);
DrawText(dc, text, text.Length, ref rect, DrawTextFlags.RightToLeft | DrawTextFlags.Right | DrawTextFlags.WordBreak);
}
finally
{
if (previousFont != IntPtr.Zero)
{
SelectObject(dc, previousFont);
}
if (hFont != IntPtr.Zero)
{
DeleteObject(hFont);
}
g.ReleaseHdc(dc);
}
}
[StructLayout(LayoutKind.Sequential)]
internal struct RECT
{
internal int Left;
internal int Top;
internal int Right;
internal int Bottom;
public static explicit operator RECT(Rectangle rect)
{
return new RECT()
{
Left = rect.Left,
Top = rect.Top,
Right = rect.Right,
Bottom = rect.Bottom
};
}
}
[DllImport("coredll.dll", CharSet = CharSet.Unicode)]
internal static extern int DrawText(IntPtr hdc, string lpStr, int nCount, ref RECT lpRect, DrawTextFlags flags);
[DllImport("coredll.dll", EntryPoint = "DeleteObject")]
[return: MarshalAs(UnmanagedType.Bool)]
internal static extern bool DeleteObject([In] IntPtr hObject);
[DllImport("coredll.dll")]
internal static extern IntPtr SelectObject(IntPtr hdc, IntPtr hgdiobj);
[Flags]
public enum DrawTextFlags : uint
{
/// <summary>
/// Use default values.
/// </summary>
None = 0x00000000,
/// <summary>
/// Justifies the text to the top of the rectangle.
/// </summary>
Top = 0x00000000,
/// <summary>
/// Aligns text to the left.
/// </summary>
Left = 0x00000000,
/// <summary>
/// Centers text horizontally in the rectangle.
/// </summary>
Center = 0x00000001,
/// <summary>
/// Aligns text to the right.
/// </summary>
Right = 0x00000002,
/// <summary>
/// Centers text vertically. This value is used only with the SingleLine value.
/// </summary>
VerticalCenter = 0x00000004,
/// <summary>
/// Justifies the text to the bottom of the rectangle. This value is used only with the
/// SingleLine value.
/// </summary>
Bottom = 0x00000008,
/// <summary>
/// Breaks words. Lines are automatically broken between words if a word would extend past the
/// edge of the rectangle specified by the lpRect parameter. A carriage return-line feed sequence
/// also breaks the line. If this is not specified, output is on one line.
/// </summary>
WordBreak = 0x00000010,
/// <summary>
/// Displays text on a single line only. Carriage returns and line feeds do not break the line.
/// </summary>
SingleLine = 0x00000020,
/// <summary>
/// Expands tab characters. The default number of characters per tab is eight.
/// </summary>
ExpandTabs = 0x00000040,
/// <summary>
/// Sets tab stops. Bits 15-8 (high-order byte of the low-order word) of the uFormat parameter
/// specify the number of characters for each tab. The default number of characters per tab is
/// eight.
/// </summary>
Tabstop = 0x00000080,
/// <summary>
/// Draws without clipping.
/// </summary>
NoClip = 0x00000100,
/// <summary>
/// Includes the font external leading in line height. Normally, external leading is not included
/// in the height of a line of text.
/// </summary>
ExternalLeading = 0x00000200,
/// <summary>
/// Determines the width and height of the rectangle. If there are multiple lines of text, DrawText
/// uses the width of the rectangle pointed to by the lpRect parameter and extends the base of the
/// rectangle to bound the last line of text. If the largest word is wider than the rectangle, the
/// width is expanded. If the text is less than the width of the rectangle, the width is reduced.
/// If there is only one line of text, DrawText modifies the right side of the rectangle so that it
/// bounds the last character in the line. In either case, DrawText returns the height of the
/// formatted text but does not draw the text.
/// </summary>
CalcRect = 0x00000400,
/// <summary>
/// Turns off processing of prefix characters. Normally, DrawText interprets the mnemonic-prefix
/// character & as a directive to underscore the character that follows, and the mnemonic-prefix
/// characters && as a directive to print a single &. By specifying DT_NOPREFIX, this processing
/// is turned off.
/// </summary>
NoPrefix = 0x00000800,
/// <summary>
/// Uses the system font to calculate text metrics.
/// </summary>
Internal = 0x00001000,
/// <summary>
/// Duplicates the text-displaying characteristics of a multiline edit control. Specifically,
/// the average character width is calculated in the same manner as for an edit control, and
/// the function does not display a partially visible last line.
/// </summary>
EditControl = 0x00002000,
/// <summary>
/// For displayed text, if the end of a string does not fit in the rectangle, it is truncated
/// and ellipses are added. If a word that is not at the end of the string goes beyond the
/// limits of the rectangle, it is truncated without ellipses.
/// </summary>
EndEllipsis = 0x00008000,
/// <summary>
/// Layout in right-to-left reading order for bidirectional text when the font selected into the
/// hdc is a Hebrew or Arabic font. The default reading order for all text is left-to-right.
/// </summary>
RightToLeft = 0x00020000,
/// <summary>
/// Truncates any word that does not fit in the rectangle and adds ellipses.
/// </summary>
WordEllipsis = 0x00040000
}
}
(As you can see I've ignored the return values of each native method)

It seems that there is no solid solution to this issue, so I created a temporary workaround. I am assuming that my Label size is fixed, and if my string size is larger than the width of my Label, I will split it into two. The method I use to split the string is to find a splitting point (currently I am doing it at 0.2 of the total string length).
private void processArabic(string arabicString)
{
string[] stringArray = new string[2];
double index = 0.8 * arabicString.Length;
index = Math.Ceiling(index);
int Index = (int)index;
for (int i = Index; i != 0; i--)
{
if (Char.IsWhiteSpace(arabicString, i))
{
stringArray[1] = arabicString.Substring(0, Index + 1);
stringArray[0] = arabicString.Substring(Index + 1, arabicString.Length - (Index + 1));
break;
}
}
label1.Text = stringArray[0];
label1.Text += "\n";
label1.Text += stringArray[1];
}
Although Arabic string is printed from right to left, index is still counted from left to right. The above snippet is rather rigid, and only apply if you want to split the string into two. I am sure there are lot of ways to improve the above code, any comment/suggestion is welcome.

How to Convert a WPF inch unit to Winforms pixels and vice versa?

I have a window that designed at WPF and I used that on center of a WinForms owner.
Now, I want to move the owner form and at the moment my WPF window has also to be moved in the center of the form!
But I have a problem, Only when window is in the center of the form that form in the center of the screen. And otherwise act in a different form than the Windows coordinates.
I just add the displacement values of the form to window location.
Now I have come to the conclusion that the coordinates of the pixels on WPF Windows are different by WinForms!
How to convert WPF window location to WinForms base location and vice versa ?
Owner Form codes is:
public partial class Form1 : Form
{
private WPF_Window.WPF win;
public Form1()
{
InitializeComponent();
win = new WPF();
win.Show();
CenterToParent(win);
}
private void CenterToParent(System.Windows.Window win)
{
win.Left = this.Left + (this.Width - win.Width) / 2;
win.Top = this.Top + (this.Height - win.Height) / 2;
}
protected override void OnMove(EventArgs e)
{
base.OnMove(e);
CenterToParent(win);
}
}

Best way to get DPI value in WPF
Method 1
It’s the same way you did that in Windows Forms. System.Drawing.Graphics object provides convenient properties to get horizontal and vertical DPI. Let’s sketch up a helper method:
/// <summary>
/// Transforms device independent units (1/96 of an inch)
/// to pixels
/// </summary>
/// <param name="unitX">a device independent unit value X</param>
/// <param name="unitY">a device independent unit value Y</param>
/// <param name="pixelX">returns the X value in pixels</param>
/// <param name="pixelY">returns the Y value in pixels</param>
public void TransformToPixels(double unitX,
double unitY,
out int pixelX,
out int pixelY)
{
using (Graphics g = Graphics.FromHwnd(IntPtr.Zero))
{
pixelX = (int)((g.DpiX / 96) * unitX);
pixelY = (int)((g.DpiY / 96) * unitY);
}
// alternative:
// using (Graphics g = Graphics.FromHdc(IntPtr.Zero)) { }
}
You can use it transforms both coordinates as well as size values. It’s pretty simple and robust and completely in managed code (at least as far as you, the consumer, is concerned). Passing IntPtr.Zero as HWND or HDC parameter results in a Graphics object that wraps a device context of the entire screen.
There is one problem with this approach though. It has a dependency on Windows Forms/GDI+ infrastructure. You are going to have to add a reference to System.Drawing assembly. Big deal? Not sure about you, but for me this is an issue to avoid.
Method 2
Let’s take it one step deeper and do it the Win API way. GetDeviceCaps function retrieves various information for the specified device and is able to retrieve horizontal and vertical DPI’s when we pass it LOGPIXELSX and LOGPIXELSY parameters respectively.
GetDeviceCaps function is defined in gdi32.dll and is probably what System.Drawing.Graphics uses under the hood.
Let’s have a look at what our helper has become:
[DllImport("gdi32.dll")]
public static extern int GetDeviceCaps(IntPtr hDc, int nIndex);
[DllImport("user32.dll")]
public static extern IntPtr GetDC(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern int ReleaseDC(IntPtr hWnd, IntPtr hDc);
public const int LOGPIXELSX = 88;
public const int LOGPIXELSY = 90;
/// <summary>
/// Transforms device independent units (1/96 of an inch)
/// to pixels
/// </summary>
/// <param name="unitX">a device independent unit value X</param>
/// <param name="unitY">a device independent unit value Y</param>
/// <param name="pixelX">returns the X value in pixels</param>
/// <param name="pixelY">returns the Y value in pixels</param>
public void TransformToPixels(double unitX,
double unitY,
out int pixelX,
out int pixelY)
{
IntPtr hDc = GetDC(IntPtr.Zero);
if (hDc != IntPtr.Zero)
{
int dpiX = GetDeviceCaps(hDc, LOGPIXELSX);
int dpiY = GetDeviceCaps(hDc, LOGPIXELSY);
ReleaseDC(IntPtr.Zero, hDc);
pixelX = (int)(((double)dpiX / 96) * unitX);
pixelY = (int)(((double)dpiY / 96) * unitY);
}
else
throw new ArgumentNullException("Failed to get DC.");
}
So we have exchanged a dependency on managed GDI+ for the dependency on fancy Win API calls. Is that an improvement? In my opinion yes, as long as we run on Windows Win API is a least common denominator. It is lightweight. On other platforms we wouldn’t probably have this dilemma in the first place.
And don’t get fooled by that ArgumentNullException. This solution is as robust as the first one. System.Drawing.Graphics will throw this same exception if it can’t obtain a device context too.
Method 3
As officially documented here there is a special key in the registry: HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontDPI. It stores a DWORD value which is exactly what the user chooses for DPI in the display settings dialog (it’s called a font size there).
Reading it is a no-brainer but I wouldn’t recommend it. You see there is a difference between an official API and a storage for various settings. The API is a public contract that stays the same even if the internal logic is totally rewritten (If it doesn’t the whole platform sucks, doesn’t it?).
But nobody guarantees that the internal storage will remain the same. It may have lasted for a couple of decades but a crucial design document that describes its relocation may already be pending an approval. You never know.
Always stick to API (whatever it is, native, Windows Forms, WPF, etc). Even if the underlying code reads the value from the location you know.
Method 4
This is a pretty elegant WPF approach that I’ve found documented in this blog post. It is based on the functionality provided by System.Windows.Media.CompositionTarget class that ultimately represents the display surface on which the WPF application is drawn. The class provides 2 useful methods:
TransformFromDevice
TransformToDevice
The names are self-explanatory and in both cases we get a System.Windows.Media.Matrix object that contains the mapping coefficients between device units (pixels) and independent units. M11 will contain a coefficient for the X axis and M22 – for the Y axis.
As we have been considering a units->pixels direction so far let’s re-write our helper with CompositionTarget.TransformToDevice. When calling this method M11 and M22 will contain values that we calculated as:
dpiX / 96
dpiY / 96
So on a machine with DPI set to 120 the coefficients will be 1.25.
Here’s the new helper:
/// <summary>
/// Transforms device independent units (1/96 of an inch)
/// to pixels
/// </summary>
/// <param name="visual">a visual object</param>
/// <param name="unitX">a device independent unit value X</param>
/// <param name="unitY">a device independent unit value Y</param>
/// <param name="pixelX">returns the X value in pixels</param>
/// <param name="pixelY">returns the Y value in pixels</param>
public void TransformToPixels(Visual visual,
double unitX,
double unitY,
out int pixelX,
out int pixelY)
{
Matrix matrix;
var source = PresentationSource.FromVisual(visual);
if (source != null)
{
matrix = source.CompositionTarget.TransformToDevice;
}
else
{
using (var src = new HwndSource(new HwndSourceParameters()))
{
matrix = src.CompositionTarget.TransformToDevice;
}
}
pixelX = (int)(matrix.M11 * unitX);
pixelY = (int)(matrix.M22 * unitY);
}
I had to add one more parameter to the method, the Visual. We need it as a base for calculations (previous samples used the device context of the entire screen for that). I don’t think it’s a big issue as you are more than likely to have a Visual at hand when running your WPF application (otherwise, why would you need to translate pixel coordinates?). However, if your visual hasn't been attached to a presentation source (that is, it hasn't been shown yet) you can't get the presentation source (thus, we have a check for NULL and construct a new HwndSource).
Reference

I've just uncovered and tested this (in VB):
formVal = Me.LogicalToDeviceUnits(WPFval)
formVal and WPFval can be either Integers or Sizes.

HUD basics using SharpDX: How to Draw over the screen surface?

Scenario
I'm planning to learn the basics about how to develop a custom HUD for a 3rd party PC-game (free, legal), my intention is to develop an application that will draw/show additional info on the screen (by reading mem addresses).
So I've researched for profesional projects and I've found TurboHUD that uses SharpDX library to draw objects/overlays/text over the screen using that DirectX based library and the results are very good (it does not loose performancy at any moment when drawing multiple objects on the screen),
but, since the author of that project does not provide a source to understand how they did it then I'm trying to learn by myself to use the same professional technics.
Question
If I'm on the wrong way or I missed some other better alternatives than SharpDX to develop this, please let me know.
My main question is:
In C# or preferably VB.Net, how I could just draw efficiently a custom string over the screen using SharpDX?.
Note that I could set the Form's oppacity to 0 but I think it should exists a proper way and I'm asking to know that proper way to draw directlly on the "desktop" screen.
My expects are to launch that PC-Game, then launch my custom HUD that will draw over the "desktop" screen surface to add additional info on the game, I hope you understand me.
Research
I should clarify that I'm totally unexperienced whit this kind of DirectX libs, and I'm using the SharpDX samples package to try learn its usage.
Since all the samples are in C# its more difficult to learn its usage under VB.Net.
Inside the SharpDX samples package there is an 'AdvancedTextRenderingApp' C# project, but as its name says it is an advanced example and also it instances a custom Form (a SharpDX.Windows.RenderForm) to draw on that Form.
This is the an VB.Net code translation of that C# project that I mentioned:
http://pastebin.com/KG2c3v09
UPDATE:
Just with the intention to comment about a research I did:
I recentlly discovered this useful GitHub repository, however, it fails to compile in Visual Studio 2015 (because missing namespace usings, that when added they generate more compiler errors), also it is oriented for advanced SharpDX users, and after analyzed the full sample I still don't have any idea of how to write/draw over the surface of a 3rd part process window ...also the C# syntax difficults me the overall understanding of SharpDX usage, because also the author did custom implementations a big variety of SharpDX members, then... I'm more than lost with all those examples.
The official SharpDX samples are another of those things that seems very useful ...maybe for advanced users. Some samples seems that demonstrates how to render a custom window/surface (with 500 tedious and incomprehensible lines of code to do it. And their "Hello world" sample is a nightmare for me.), however, what I would like to acchieve as I explained in my question is to draw on the surface of an existing window of another process, and I'm aware that probablly for that I would need to render a "surface" from scratch with SharpDX, then positionate it in the target window, then make invisible the surface, then draw on it, but I don't know how to do those things.

It took me a while to find how to load the font in XNA to draw the text but everything works fine.
You need 4 things:
Make form topmost
Extend aero glass style to whole form (for transparency to work)
Initialize XNA, Microsoft XNA Game Studio 4.0
Draw the texture and text
One limitation
The game must not be in fullscreen mode. The same as the TurboHUD limitation
Form1.cs
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Text;
using System.IO;
using System.Windows.Forms;
using System.Threading;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Content;
using System.Runtime.InteropServices;
using System.Drawing;
namespace XNATransparentWindow
{
public partial class Form1 : Form
{
private ContentBuilder contentBuilder;
public Form1()
{
InitializeComponent();
TopMost = true;
FormBorderStyle = System.Windows.Forms.FormBorderStyle.None;
MARGINS margins = new MARGINS();
margins.leftWidth = 0;
margins.rightWidth = 0;
margins.topHeight = this.Width;
margins.bottomHeight = this.Height;
// Extend aero glass style to whole form
DwmExtendFrameIntoClientArea(this.Handle, ref margins);
//Load XNA directX
this.contentBuilder = new ContentBuilder();
graphicsDeviceService = GraphicsDeviceService.AddRef(Handle, ClientSize.Width, ClientSize.Height);
//Register the service, so components like ContentManager can find it.
services.AddService<IGraphicsDeviceService>(graphicsDeviceService);
//Get the Graphics device
dev = graphicsDeviceService.GraphicsDevice;
if(dev == null){/*error message*/}
//Load texture
int bufferSize;
System.IO.MemoryStream memoryStream;
Bitmap img;
using (img = new Bitmap(#"C:\...\.png"))
{
bufferSize = img.Height * img.Width * 4;
memoryStream = new System.IO.MemoryStream(bufferSize);
img.Save(memoryStream, System.Drawing.Imaging.ImageFormat.Png);
memoryStream.Seek(0, SeekOrigin.Begin);
texture = Texture2D.FromStream(dev, memoryStream, img.Width, img.Height, false);
memoryStream.Close();
if(texture == null){/*error message*/}
}
//Create sprite
s_Batch = new SpriteBatch(dev);
if(s_Batch == null){/*error message*/}
FontPos = new Vector2(270.0F, 110.0F);
//Load font
contentManager = new ContentManager(services, this.contentBuilder.OutputDirectory);
this.contentBuilder.Clear();
this.contentBuilder.Add(#"C:\...\my_font1.spritefont","my_font1", "FontDescriptionImporter", "FontDescriptionProcessor");
//Build spritefont to get the .xbn file
string error = this.contentBuilder.Build();
//If build fail
if (!String.IsNullOrEmpty(error))
{
MessageBox.Show(error);
return;
}
//Load the .xbn file
Font1 = contentManager.Load<SpriteFont>("my_font1");
if(Font1 == null){/*error message*/}
}
[DllImport("dwmapi.dll")]
static extern int DwmExtendFrameIntoClientArea(IntPtr hWnd, ref MARGINS margin);
[DllImport("user32.dll")]
static extern IntPtr SendMessage(IntPtr hWnd, UInt32 Msg, Int32 wParam, Int32 lParam);
[StructLayout(LayoutKind.Sequential)]
public struct MARGINS
{
public int leftWidth;
public int rightWidth;
public int topHeight;
public int bottomHeight;
}
public ServiceContainer Services
{
get { return services; }
}
ServiceContainer services = new ServiceContainer();
GraphicsDevice dev;
SpriteFont Font1;
Vector2 FontPos;
SpriteBatch s_Batch;
Texture2D texture;
ContentManager contentManager;
GraphicsDeviceService graphicsDeviceService;
private const UInt32 WM_NCLBUTTONDOWN = 0xA1;
private const Int32 HTCAPTION = 0x2;
private void Form1_MouseDown(object sender, MouseEventArgs e)
{
if (e.Button == System.Windows.Forms.MouseButtons.Right)
{
this.Close();
}
else //to move the form
{
this.Capture = false;
SendMessage(this.Handle, WM_NCLBUTTONDOWN, HTCAPTION, 0);
}
}
private void Form1_Paint(object sender, PaintEventArgs e)
{
//There are two buffers. One offscreen-backbuffer and the
//frontbuffer which is the actual form in this example. The
//drawings are done to the backbuffer and in the end the two
//buffers flip. The backbuffer becomes frontbuffer and the
//frontbuffer becomes backbuffer.
//The drawing should start when the last resource is loaded.
//Since Font1 is the last one in this example I check for this
if (Font1 == null)
{
return;
}
//clear the backbuffer with transparent color.
dev.Clear(Microsoft.Xna.Framework.Color.Transparent);
//Do all your drawings here
//draw texture and text with the sprite
s_Batch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend);
s_Batch.Draw(texture, new Microsoft.Xna.Framework.Rectangle(0, 0, this.Width, this.Height), Microsoft.Xna.Framework.Color.White);
s_Batch.DrawString(Font1, #"XNA FRAMEWORK", FontPos, Microsoft.Xna.Framework.Color.Black);
s_Batch.End();
//here the flip is performed
dev.Present();
}
//Release resources
private void Form1_FormClosing(object sender, FormClosingEventArgs e)
{
graphicsDeviceService.GraphicsDevice.Dispose();
graphicsDeviceService.Release(true);
s_Batch.Dispose();
texture.Dispose();
}
}
}
The following classes are just copy-paste from an example XNA 4.0 Content Compiler I found(with minor adjustments). They are used just to load the font for drawing the text:
GraphicsDeviceService.cs
#region File Description
//-----------------------------------------------------------------------------
// GraphicsDeviceService.cs
//
// Microsoft XNA Community Game Platform
// Copyright (C) Microsoft Corporation. All rights reserved.
//-----------------------------------------------------------------------------
#endregion
#region Using Statements
using System;
using System.Threading;
using Microsoft.Xna.Framework.Graphics;
#endregion
// The IGraphicsDeviceService interface requires a DeviceCreated event, but we
// always just create the device inside our constructor, so we have no place to
// raise that event. The C# compiler warns us that the event is never used, but
// we don't care so we just disable this warning.
#pragma warning disable 67
namespace XNATransparentWindow
{
/// <summary>
/// Helper class responsible for creating and managing the GraphicsDevice.
/// All GraphicsDeviceControl instances share the same GraphicsDeviceService,
/// so even though there can be many controls, there will only ever be a single
/// underlying GraphicsDevice. This implements the standard IGraphicsDeviceService
/// interface, which provides notification events for when the device is reset
/// or disposed.
/// </summary>
class GraphicsDeviceService : IGraphicsDeviceService
{
#region Fields
// Singleton device service instance.
static GraphicsDeviceService singletonInstance;
// Keep track of how many controls are sharing the singletonInstance.
static int referenceCount;
#endregion
/// <summary>
/// Constructor is private, because this is a singleton class:
/// client controls should use the public AddRef method instead.
/// </summary>
GraphicsDeviceService(IntPtr windowHandle, int width, int height)
{
parameters = new PresentationParameters();
parameters.BackBufferWidth = Math.Max(width, 1);
parameters.BackBufferHeight = Math.Max(height, 1);
parameters.BackBufferFormat = SurfaceFormat.Vector4; // SurfaceFormat.Color;
parameters.DeviceWindowHandle = windowHandle;
parameters.PresentationInterval = PresentInterval.Immediate;
parameters.IsFullScreen = false;
graphicsDevice = new GraphicsDevice(GraphicsAdapter.DefaultAdapter, GraphicsProfile.Reach, parameters);
}
/// <summary>
/// Gets a reference to the singleton instance.
/// </summary>
public static GraphicsDeviceService AddRef(IntPtr windowHandle,
int width, int height)
{
// Increment the "how many controls sharing the device" reference count.
if (Interlocked.Increment(ref referenceCount) == 1)
{
// If this is the first control to start using the
// device, we must create the singleton instance.
singletonInstance = new GraphicsDeviceService(windowHandle,
width, height);
}
return singletonInstance;
}
/// <summary>
/// Releases a reference to the singleton instance.
/// </summary>
public void Release(bool disposing)
{
// Decrement the "how many controls sharing the device" reference count.
if (Interlocked.Decrement(ref referenceCount) == 0)
{
// If this is the last control to finish using the
// device, we should dispose the singleton instance.
if (disposing)
{
if (DeviceDisposing != null)
DeviceDisposing(this, EventArgs.Empty);
graphicsDevice.Dispose();
}
graphicsDevice = null;
}
}
/// <summary>
/// Resets the graphics device to whichever is bigger out of the specified
/// resolution or its current size. This behavior means the device will
/// demand-grow to the largest of all its GraphicsDeviceControl clients.
/// </summary>
public void ResetDevice(int width, int height)
{
if (DeviceResetting != null)
DeviceResetting(this, EventArgs.Empty);
parameters.BackBufferWidth = Math.Max(parameters.BackBufferWidth, width);
parameters.BackBufferHeight = Math.Max(parameters.BackBufferHeight, height);
graphicsDevice.Reset(parameters);
if (DeviceReset != null)
DeviceReset(this, EventArgs.Empty);
}
/// <summary>
/// Gets the current graphics device.
/// </summary>
public GraphicsDevice GraphicsDevice
{
get { return graphicsDevice; }
}
GraphicsDevice graphicsDevice;
// Store the current device settings.
PresentationParameters parameters;
// IGraphicsDeviceService events.
public event EventHandler<EventArgs> DeviceCreated;
public event EventHandler<EventArgs> DeviceDisposing;
public event EventHandler<EventArgs> DeviceReset;
public event EventHandler<EventArgs> DeviceResetting;
}
}
ServiceContainer.cs
#region File Description
//-----------------------------------------------------------------------------
// ServiceContainer.cs
//
// Microsoft XNA Community Game Platform
// Copyright (C) Microsoft Corporation. All rights reserved.
//-----------------------------------------------------------------------------
#endregion
#region Using Statements
using System;
using System.Collections.Generic;
#endregion
namespace XNATransparentWindow
{
/// <summary>
/// Container class implements the IServiceProvider interface. This is used
/// to pass shared services between different components, for instance the
/// ContentManager uses it to locate the IGraphicsDeviceService implementation.
/// </summary>
public class ServiceContainer : IServiceProvider
{
Dictionary<Type, object> services = new Dictionary<Type, object>();
/// <summary>
/// Adds a new service to the collection.
/// </summary>
public void AddService<T>(T service)
{
services.Add(typeof(T), service);
}
/// <summary>
/// Looks up the specified service.
/// </summary>
public object GetService(Type serviceType)
{
object service;
services.TryGetValue(serviceType, out service);
return service;
}
}
}
ContentBuilder.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Diagnostics;
using Microsoft.Build.Construction;
using Microsoft.Build.Evaluation;
using Microsoft.Build.Execution;
using Microsoft.Build.Framework;
namespace XNATransparentWindow
{
public class ContentBuilder : IDisposable
{
#region Fields
// What importers or processors should we load?
const string xnaVersion = ", Version=4.0.0.0, PublicKeyToken=842cf8be1de50553";
static string[] pipelineAssemblies =
{
"Microsoft.Xna.Framework.Content.Pipeline.FBXImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.XImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.TextureImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.EffectImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.AudioImporters" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.VideoImporters" + xnaVersion,
// If you want to use custom importers or processors from
// a Content Pipeline Extension Library, add them here.
//
// If your extension DLL is installed in the GAC, you should refer to it by assembly
// name, eg. "MyPipelineExtension, Version=1.0.0.0, PublicKeyToken=1234567812345678".
//
// If the extension DLL is not in the GAC, you should refer to it by
// file path, eg. "c:/MyProject/bin/MyPipelineExtension.dll".
};
// MSBuild objects used to dynamically build content.
Project buildProject;
ProjectRootElement projectRootElement;
BuildParameters buildParameters;
List<ProjectItem> projectItems = new List<ProjectItem>();
//ErrorLogger errorLogger;
// Temporary directories used by the content build.
string buildDirectory;
string processDirectory;
string baseDirectory;
// Generate unique directory names if there is more than one ContentBuilder.
static int directorySalt;
// Have we been disposed?
bool isDisposed;
#endregion
#region Properties
/// Gets the output directory, which will contain the generated .xnb files.
public string OutputDirectory
{
get { return Path.Combine(buildDirectory, "bin/Content"); }
}
#endregion
#region Initialization
/// Creates a new content builder.
public ContentBuilder()
{
CreateTempDirectory();
CreateBuildProject();
}
/// Finalizes the content builder.
~ContentBuilder()
{
Dispose(false);
}
/// Disposes the content builder when it is no longer required.
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
/// Implements the standard .NET IDisposable pattern.
protected virtual void Dispose(bool disposing)
{
if (!isDisposed)
{
isDisposed = true;
DeleteTempDirectory();
}
}
#endregion
#region MSBuild
/// Creates a temporary MSBuild content project in memory.
void CreateBuildProject()
{
string projectPath = Path.Combine(buildDirectory, "content.contentproj");
string outputPath = Path.Combine(buildDirectory, "bin");
// Create the build project.
projectRootElement = ProjectRootElement.Create(projectPath);
// Include the standard targets file that defines how to build XNA Framework content.
projectRootElement.AddImport("$(MSBuildExtensionsPath)\\Microsoft\\XNA Game Studio\\" +
"v4.0\\Microsoft.Xna.GameStudio.ContentPipeline.targets");
buildProject = new Project(projectRootElement);
buildProject.SetProperty("XnaPlatform", "Windows");
buildProject.SetProperty("XnaProfile", "Reach");
buildProject.SetProperty("XnaFrameworkVersion", "v4.0");
buildProject.SetProperty("Configuration", "Release");
buildProject.SetProperty("OutputPath", outputPath);
// Register any custom importers or processors.
foreach (string pipelineAssembly in pipelineAssemblies)
{
buildProject.AddItem("Reference", pipelineAssembly);
}
// Hook up our custom error logger.
//errorLogger = new ErrorLogger();
buildParameters = new BuildParameters(ProjectCollection.GlobalProjectCollection);
//buildParameters.Loggers = new ILogger[] { errorLogger };
}
/// Adds a new content file to the MSBuild project. The importer and
/// processor are optional: if you leave the importer null, it will
/// be autodetected based on the file extension, and if you leave the
/// processor null, data will be passed through without any processing.
public void Add(string filename, string name, string importer, string processor)
{
ProjectItem item = buildProject.AddItem("Compile", filename)[0];
item.SetMetadataValue("Link", Path.GetFileName(filename));
item.SetMetadataValue("Name", name);
if (!string.IsNullOrEmpty(importer))
item.SetMetadataValue("Importer", importer);
if (!string.IsNullOrEmpty(processor))
item.SetMetadataValue("Processor", processor);
projectItems.Add(item);
}
/// Removes all content files from the MSBuild project.
public void Clear()
{
buildProject.RemoveItems(projectItems);
projectItems.Clear();
}
/// Builds all the content files which have been added to the project,
/// dynamically creating .xnb files in the OutputDirectory.
/// Returns an error message if the build fails.
public string Build()
{
// Create and submit a new asynchronous build request.
BuildManager.DefaultBuildManager.BeginBuild(buildParameters);
BuildRequestData request = new BuildRequestData(buildProject.CreateProjectInstance(), new string[0]);
BuildSubmission submission = BuildManager.DefaultBuildManager.PendBuildRequest(request);
submission.ExecuteAsync(null, null);
// Wait for the build to finish.
submission.WaitHandle.WaitOne();
BuildManager.DefaultBuildManager.EndBuild();
// If the build failed, return an error string.
if (submission.BuildResult.OverallResult == BuildResultCode.Failure)
{
//return string.Join("\n", errorLogger.Errors.ToArray());
}
return null;
}
#endregion
#region Temp Directories
/// Creates a temporary directory in which to build content.
void CreateTempDirectory()
{
// Start with a standard base name:
//
// %temp%\WinFormsContentLoading.ContentBuilder
baseDirectory = Path.Combine(Path.GetTempPath(), GetType().FullName);
// Include our process ID, in case there is more than
// one copy of the program running at the same time:
//
// %temp%\WinFormsContentLoading.ContentBuilder\<ProcessId>
int processId = Process.GetCurrentProcess().Id;
processDirectory = Path.Combine(baseDirectory, processId.ToString());
// Include a salt value, in case the program
// creates more than one ContentBuilder instance:
//
// %temp%\WinFormsContentLoading.ContentBuilder\<ProcessId>\<Salt>
directorySalt++;
buildDirectory = Path.Combine(processDirectory, directorySalt.ToString());
// Create our temporary directory.
Directory.CreateDirectory(buildDirectory);
PurgeStaleTempDirectories();
}
/// <summary>
/// Deletes our temporary directory when we are finished with it.
/// </summary>
void DeleteTempDirectory()
{
Directory.Delete(buildDirectory, true);
// If there are no other instances of ContentBuilder still using their
// own temp directories, we can delete the process directory as well.
if (Directory.GetDirectories(processDirectory).Length == 0)
{
Directory.Delete(processDirectory);
// If there are no other copies of the program still using their
// own temp directories, we can delete the base directory as well.
if (Directory.GetDirectories(baseDirectory).Length == 0)
{
Directory.Delete(baseDirectory);
}
}
}
/// <summary>
/// Ideally, we want to delete our temp directory when we are finished using
/// it. The DeleteTempDirectory method (called by whichever happens first out
/// of Dispose or our finalizer) does exactly that. Trouble is, sometimes
/// these cleanup methods may never execute. For instance if the program
/// crashes, or is halted using the debugger, we never get a chance to do
/// our deleting. The next time we start up, this method checks for any temp
/// directories that were left over by previous runs which failed to shut
/// down cleanly. This makes sure these orphaned directories will not just
/// be left lying around forever.
/// </summary>
void PurgeStaleTempDirectories()
{
// Check all subdirectories of our base location.
foreach (string directory in Directory.GetDirectories(baseDirectory))
{
// The subdirectory name is the ID of the process which created it.
int processId;
if (int.TryParse(Path.GetFileName(directory), out processId))
{
try
{
// Is the creator process still running?
Process.GetProcessById(processId);
}
catch (ArgumentException)
{
// If the process is gone, we can delete its temp directory.
Directory.Delete(directory, true);
}
}
}
}
#endregion
}
}
The font which you are loading is an xml file: any_name.spritefont
Example:
<?xml version="1.0" encoding="utf-8"?>
<!--
This file contains an xml description of a font, and will be read by the XNA
Framework Content Pipeline. Follow the comments to customize the appearance
of the font in your game, and to change the characters which are available to draw
with.
-->
<XnaContent xmlns:Graphics="Microsoft.Xna.Framework.Content.Pipeline.Graphics">
<Asset Type="Graphics:FontDescription">
<!--
Modify this string to change the font that will be imported.
-->
<FontName>Segoe UI Mono</FontName>
<!--
Size is a float value, measured in points. Modify this value to change
the size of the font.
-->
<Size>14</Size>
<!--
Spacing is a float value, measured in pixels. Modify this value to change
the amount of spacing in between characters.
-->
<Spacing>0</Spacing>
<!--
UseKerning controls the layout of the font. If this value is true, kerning information
will be used when placing characters.
-->
<UseKerning>true</UseKerning>
<!--
Style controls the style of the font. Valid entries are "Regular", "Bold", "Italic",
and "Bold, Italic", and are case sensitive.
-->
<Style>Regular</Style>
<!--
If you uncomment this line, the default character will be substituted if you draw
or measure text that contains characters which were not included in the font.
-->
<!-- <DefaultCharacter>*</DefaultCharacter> -->
<!--
CharacterRegions control what letters are available in the font. Every
character from Start to End will be built and made available for drawing. The
default range is from 32, (ASCII space), to 126, ('~'), covering the basic Latin
character set. The characters are ordered according to the Unicode standard.
See the documentation for more information.
-->
<CharacterRegions>
<CharacterRegion>
<Start> </Start>
<End>~</End>
</CharacterRegion>
</CharacterRegions>
</Asset>
</XnaContent>
If you want the form to be click-through then you need to add a random
transparency key color to form and:
[DllImport("user32.dll", EntryPoint="GetWindowLong")]
static extern IntPtr GetWindowLongPtr(IntPtr hWnd, int nIndex);
[DllImport("user32.dll")]
static extern int SetWindowLong(IntPtr hWnd, int nIndex, IntPtr dwNewLong);
In public Form1()
IntPtr initialStyle = GetWindowLongPtr(this.Handle, -20);
SetWindowLong(this.Handle, -20, (IntPtr)((int)initialStyle | 0x20));
Feel free to correct any mistakes I have made.

.NET and Direct2D rapid draw crash

General purpose
I am developing a .NET user control that can be used for multipurpose graphics rendering using Direct2D. This control is based on the ID2D1HwndRenderTarget interface with a ID2D1BitmapRenderTarget back-buffer/double-buffer.
Background
I have written a wrapper using C++/CLI that exposes a set of managed wrapper objects for Direct2D, and I have a designed a Singleton resource manager for Direct2D bitmaps that code can send to the binary data to, and get a resource key back from, which code can then pass in a message to the user control to render a bitmap.
Usage so far
I have been able to load bitmaps, JPEGs, custom imaging formats and the like and send them to the Direct2D render control all with no issue. I've written a custom video decoder that could load frames from a 15-fps video (decoding in a background thread) and render them from events raised by a Win32 multimedia timer, no problem.
Issue
My issue is that when I attempted to expand out from a single multimedia format into something more flexible (specifically connecting to LibAV/ffmpeg), the Direct2D render control starts to crash display drivers. This does not happen when frames are rendered sequentially (using a button to render the next frame, rather than a timer). It also does not happen straightaway. It also does not happen when I block inside the Win32 timer callback, but does when I use a Mutex to raise a thread that will free the timer callback up and let the interim thread instruct the control to render.
Symptoms
If I start a timer and attach to its elapsed event methods that will cause the next render, it will typically play fine for around 2-20 seconds. Using smaller video, I can get longer playback before the issue starts. If I play back 1080p video, I can get the crash usually within 5 seconds without contest. Playback will start fine, maybe have a hiccup here or there before catching up.
Eventually, I will start to see flickering, as if the background color is rendered but the frame is not. Additional frames may or may not later render, but this is very short-lived.
After the flicker, if I stop the timer quickly enough there will be no crash.
If I do not stop in time:
If I'm lucky (maybe 10% of the time), frames will stop rendering, be stuck on one frame until I resize the control, at which point only black is drawn to the entire control. (I've read that this may indicate a lost device context for DirectX rendering, but have not seen much more.)
If I'm unlucky, the display driver will crash, Windows recovers and restarts the driver, after which point the EndDraw will finally tell me that an error has occurred and return D2DERR_RECREATE_TARGET.
If I am really unlucky, the display will start to look like a kaleidoscope and I'll have to power off my machine, and I just wasted 5 minutes booting, logging on, loading Visual Studio and my solution, loading the video and lost all my debugging data.
I want to think that there is some sort of race condition that I am missing, but every time I run through the rendering code, it appears that it should be properly locked.
Render Control's Code
using System;
using System.Drawing;
using System.Windows.Forms;
using Direct2D = Bardez.Projects.DirectX.Direct2D;
using ExternalPixelEnums = Bardez.Projects.FileFormats.MediaBase.Video.Pixels.Enums;
using Bardez.Projects.DirectX.Direct2D;
using Bardez.Projects.FileFormats.MediaBase.Video;
using Bardez.Projects.FileFormats.MediaBase.Video.Enums;
using Bardez.Projects.Win32;
namespace Bardez.Projects.Output.Visual
{
/// <summary>Represents a rendering target for Direct2D.</summary>
/// <remarks>
/// To use this control, an external component will need to supply bitmap data. This does not need to be a GDI+ Bitmap class.
/// However, the container for this control will need to push data into this control.
/// So, in the case of a movie player, we'd see the following model:
/// * decompress a frame.
/// * push frame to control
/// * invoke Invalidate
/// * control will render the bitmap
/// * sleep just a little bit
/// * go back to first step
/// </remarks>
public class Direct2dRenderControl : VisualRenderControl
{
/*
* Locking orientation:
* There are two rendering targets: a GDI display and a bitmap back buffer.
* There are 5 'real' locking operations:
* Rendering to the GDI display (OnPaint)
* Rendering to the back buffer (DrawBitmapToBuffer, DiscardCurrentBuffer)
* Setting the current displayed frame (SetRenderFrame)
* Resource Freeing (FreeFrameResource)
* Resizing (OnResize)
*
* Briefly, the overarching effects of these five are:
* Rendering
* Utilizes the buffer and the display
* Back Buffer
* Utilizes the buffer and if mid-render could affect this display
* Set Frame
* Sets the buffer's displayed image
* Resource Freeing
* Affects the buffer if a resource is refernced
* Resizing
* Resizes the render control and the bitmap, uses the frame set by set
*
* Locking plan:
* Resize should block set, free and back buffer
* Render should block back buffer, control
* Set frame should block resize and back buffer
* Free should block back buffer,
* Back buffer should block rendering, setting, resizing, freeing
*
* Basically, lock everything at the process level, and not at the access level.
*/
#region Fields
/// <summary>Represents a Win32 HWND render target for Direct2D</summary>
private ControlRenderTarget ctrlRenderTarget;
/// <summary>Represents a drawing buffer</summary>
private BitmapRenderTarget bmpRenderTarget;
/// <summary>Represents a drawing buffer, used solely to create bitmaps</summary>
private BitmapRenderTarget resourceBmpRenderTarget;
/// <summary>Represents a buffer drawing command lock</summary>
private Object controlBufferLock;
/// <summary>Represents a paint/render drawing command lock</summary>
private Object controlPaintRenderLock;
/// <summary>Locking object reference for resource management (memory bitmaps, etc.)</summary>
private Object resourceLock;
/// <summary>Represents the key to accessing the currently set key</summary>
protected Int32 currentFrameKey;
#endregion
#region Properties
/// <summary>Indicates whether there is a frame set for this control</summary>
protected Boolean HasFrameSet
{
get { return currentFrameKey > -1; }
}
/// <summary>Exposes a wrapper for the bitmap render target</summary>
protected BitmapRenderTarget BmpRenderTarget
{
get { return this.bmpRenderTarget; }
set
{
lock (this.controlBufferLock)
{
if (this.bmpRenderTarget != null)
this.bmpRenderTarget.Dispose();
this.bmpRenderTarget = value;
}
}
}
/// <summary>Exposes a wrapper for the bitmap render target</summary>
protected BitmapRenderTarget ResourceBmpRenderTarget
{
get { return this.resourceBmpRenderTarget; }
set
{
lock (this.resourceLock)
{
if (this.resourceBmpRenderTarget != null)
this.resourceBmpRenderTarget.Dispose();
this.resourceBmpRenderTarget = value;
}
}
}
/// <summary>Represents a Win32 HWND render target for Direct2D</summary>
protected ControlRenderTarget CtrlRenderTarget
{
get { return this.ctrlRenderTarget; }
set
{
lock (this.controlPaintRenderLock)
{
if (this.ctrlRenderTarget != null)
this.ctrlRenderTarget.Dispose();
this.ctrlRenderTarget = value;
}
}
}
#endregion
#region Construction
/// <summary>Default constructor</summary>
public Direct2dRenderControl() : base()
{
this.controlBufferLock = new Object();
this.controlPaintRenderLock = new Object();
this.resourceLock = new Object();
this.currentFrameKey = -1;
this.InitializeControlDirect2D();
}
/// <summary>Initializes the Direct2D</summary>
protected void InitializeControlDirect2D()
{
//disable Windows background draw
this.SetStyle(ControlStyles.UserPaint | ControlStyles.AllPaintingInWmPaint, true);
// Build options on the Control render target
PixelFormat format = new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.Unknown); //32-bit color, pure alpha
DpiResolution res = Direct2dResourceManager.Instance.Factory.GetDesktopDpi();
RenderTargetProperties rtProp = new RenderTargetProperties(RenderTargetType.Default, format, res, RenderTargetUsage.GdiCompatible, DirectXVersion.DirectX9);
//Build out control render target properties
HwndRenderTargetProperties hwndProp = new HwndRenderTargetProperties(this.Handle, new SizeU(this.Size), PresentationOptions.RetainContents);
lock (this.controlPaintRenderLock)
{
// populate the Control rendering target
ResultCode result = Direct2dResourceManager.Instance.Factory.CreateHwndRenderTarget(rtProp, hwndProp, out this.ctrlRenderTarget);
lock (this.controlBufferLock)
{
// create a bitmap rendering targets
this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.bmpRenderTarget);
lock (this.resourceLock)
this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.resourceBmpRenderTarget);
}
}
}
#endregion
#region Destruction
/// <summary>Disposal code; releases unmanaged resources</summary>
/// <param name="disposing">True indicates to dispose managed resources</param>
protected override void Dispose(Boolean disposing)
{
this.ResourceBmpRenderTarget = null; //property disposes
this.BmpRenderTarget = null; //property disposes
this.CtrlRenderTarget = null; //property disposes
base.Dispose(disposing);
}
/// <summary>Disposal</summary>
~Direct2dRenderControl()
{
this.Dispose();
}
#endregion
#region Event Raising
/// <summary>Draws the output, then raises the paint event</summary>
/// <param name="e">Painting Event arguments</param>
protected override void OnPaint(PaintEventArgs e)
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
this.SuspendLayout();
this.OnPaintBackground(e);
Direct2D.Bitmap bmp;
ResultCode result;
result = this.BmpRenderTarget.GetBitmap(out bmp);
Direct2D.RectangleF rect = new Direct2D.RectangleF(e.ClipRectangle);
this.CtrlRenderTarget.BeginDraw();
this.CtrlRenderTarget.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);
result = this.CtrlRenderTarget.EndDraw();
bmp.Dispose();
if (result != ResultCode.Success_OK)
throw new ApplicationException(String.Format("Error encountered during draw: '{0}'", result.ToString()));
base.OnPaint(e);
this.ResumeLayout();
}
}
}
/// <summary>Overides the resize method</summary>
/// <param name="e">Parameters for the resize event</param>
protected override void OnResize(EventArgs e)
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
//Null check since resize fires before it is constructed, with the size event during parent's InitializeComponent
if (this.CtrlRenderTarget != null)
{
this.SuspendLayout();
//resize the existing control rendering target
this.CtrlRenderTarget.Resize(new SizeU(this.Size));
//I also need to resize the buffer, but can't. Instead, create a new one, then copy the existing one. Kind of lame.
this.ResizeBitmapRenderTarget();
base.OnResize(e);
//cause another draw
this.Invalidate(new Rectangle(new Point(0, 0), this.Size));
this.ResumeLayout();
}
}
}
}
/// <summary>Overridden Paint background method</summary>
/// <param name="e">Paint event arguments</param>
/// <remarks>
/// Made empty to avoid GDI and Direct2D writing to the same control; when moving the control around
/// or rendering at high speeds, the dreaded WinForms flicker was introduced.
/// The background painting can be found in the <see cref="FillBufferRenderTarget"/> method,
/// which fills the bitmap back buffer with the control's background color.
/// </remarks>
protected override void OnPaintBackground(PaintEventArgs e) { }
#endregion
#region Drawing
/// <summary>Resizes the bitmap rendering target buffer</summary>
/// <remarks>Does not lock the GDI render target. The OnResize or other callers lock instead.</remarks>
protected void ResizeBitmapRenderTarget()
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
using (BitmapRenderTarget bmpCurr = this.BmpRenderTarget)
{
BitmapRenderTarget bmpNew;
ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);
this.DuplicateDoubleBufferContents(bmpNew);
//Property disposes and locks
this.BmpRenderTarget = bmpNew;
}
}
}
}
/// <summary>Draws a Bitmap to the render buffer</summary>
/// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp, Point2dF origin)
{
lock (this.controlBufferLock)
{
lock (this.resourceLock)
{
Direct2D.SizeF bmpSize = bmp.GetSize();
Single width = bmpSize.Width > this.BmpRenderTarget.Size.Width ? this.BmpRenderTarget.Size.Width : bmpSize.Width;
Single height = bmpSize.Height > this.BmpRenderTarget.Size.Height ? this.BmpRenderTarget.Size.Height : bmpSize.Height;
Direct2D.RectangleF destRect = new Direct2D.RectangleF(origin.X, origin.X + width, origin.Y, origin.Y + height);
Direct2D.RectangleF srcRect = new Direct2D.RectangleF(0.0F, width, 0.0F, height);
this.BmpRenderTarget.BeginDraw();
this.FillBufferRenderTarget(this.BmpRenderTarget);
// do the actual draw
this.BmpRenderTarget.DrawBitmap(bmp, destRect, 1.0F, BitmapInterpolationMode.Linear, srcRect);
//tell Direct2D that a paint operation is ending
ResultCode result = this.BmpRenderTarget.EndDraw();
}
}
}
/// <summary>Draws a Bitmap to the render buffer</summary>
/// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp)
{
this.DrawBitmapToBuffer(bmp, new Point2dF(0.0F, 0.0F));
}
/// <summary>Duplicates the bitmap behind the existing rendering target, and drawing it to a new one, discarding the current and setting the new.</summary>
/// <remarks>Does not lock any references, as the outside method locks</remarks>
protected void DuplicateDoubleBufferContents(BitmapRenderTarget bmpNew)
{
Direct2D.Bitmap bmp = null;
ResultCode result = ResultCode.Success_OK;
if (this.HasFrameSet)
bmp = Direct2dResourceManager.Instance.GetBitmapResource(this.currentFrameKey);
else
result = this.BmpRenderTarget.GetBitmap(out bmp);
bmpNew.BeginDraw();
this.FillBufferRenderTarget(bmpNew);
//calculate the size to copy
Direct2D.SizeF bmpSize = bmp.GetSize();
Single width = bmpSize.Width > this.CtrlRenderTarget.Size.Width ? this.CtrlRenderTarget.Size.Width : bmpSize.Width;
Single height = bmpSize.Height > this.CtrlRenderTarget.Size.Height ? this.CtrlRenderTarget.Size.Height : bmpSize.Height;
//Determine the copy rectangle
Direct2D.RectangleF rect = new Direct2D.RectangleF(0, width, 0, height);
//Copy
bmpNew.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);
//conditionally disose the bitmap, don't if it is in the manager
if (!this.HasFrameSet)
bmp.Dispose();
result = bmpNew.EndDraw();
}
/// <summary>Discards the current buffer and replaces it with a blank one.</summary>
protected void DiscardCurrentBuffer()
{
lock (this.controlBufferLock)
{
BitmapRenderTarget bmpNew;
// create a bitmap rendering target
ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);
bmpNew.BeginDraw();
this.FillBufferRenderTarget(bmpNew);
result = bmpNew.EndDraw();
//property locks, so no lock here
this.BmpRenderTarget = bmpNew; //replace the old buffer
}
}
/// <summary>Fills the buffer render target with the background color</summary>
/// <param name="renderTarget">Bitmap render target to fill</param>
protected void FillBufferRenderTarget(BitmapRenderTarget renderTarget)
{
SolidColorBrush brush = null;
ResultCode result = renderTarget.CreateSolidColorBrush(new ColorF(this.BackColor), out brush);
renderTarget.FillRectangle(new Direct2D.RectangleF(new Rectangle(new Point(0, 0), this.Size)), brush);
brush.Dispose();
}
#endregion
#region Frame Resources
/// <summary>Posts a Frame resource to the resource manager and returns a unique key to access it.</summary>
/// <param name="resource">Frame to be posted.</param>
/// <returns>A unique Int32 key</returns>
public override Int32 AddFrameResource(Frame resource)
{
lock (this.resourceLock)
{
//create the bitmap
BitmapProperties properties = new BitmapProperties(new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.PreMultiplied), Direct2dResourceManager.Instance.Factory.GetDesktopDpi());
SizeU dimensions = new SizeU(Convert.ToUInt32(resource.Pixels.Metadata.Width), Convert.ToUInt32(resource.Pixels.Metadata.Height));
Direct2D.Bitmap bmp = null;
ResultCode result = this.ResourceBmpRenderTarget.CreateBitmap(dimensions, properties, out bmp);
Byte[] data = resource.Pixels.GetPixelData(ExternalPixelEnums.PixelFormat.RGBA_B8G8R8A8, ScanLineOrder.TopDown, 0, 0);
result = bmp.CopyFromMemory(new RectangleU(dimensions), data, Convert.ToUInt32(resource.Pixels.Metadata.Width * 4));
return Direct2dResourceManager.Instance.AddFrameResource(bmp);
}
}
/// <summary>Frees a Bitmap resource in the resource manager and Disposes of it.</summary>
/// <param name="frameKey">Direct2D Bitmap key to be Disposed.</param>
public override void FreeFrameResource(Int32 frameKey)
{
lock (this.resourceLock)
{
if (frameKey > -1)
Direct2dResourceManager.Instance.FreeFrameResource(frameKey);
}
}
#endregion
#region Frame Setting
/// <summary>Sets the frame to be rendered to the User Control</summary>
/// <param name="key">Frame key to set as current image</param>
public override void SetRenderFrame(Int32 key)
{
this.SetRenderFrame(key, 0, 0);
}
/// <summary>Sets the frame to be rendered to the User Control</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
public override void SetRenderFrame(Int32 key, Int64 originX, Int64 originY)
{
lock (this.controlBufferLock)
{
if (key > -1)
this.DrawBitmapToBuffer(Direct2dResourceManager.Instance.GetBitmapResource(key), new Point2dF(Convert.ToSingle(originX), Convert.ToSingle(originY)));
else
this.DiscardCurrentBuffer();
this.currentFrameKey = key;
}
}
/// <summary>This method invokes the rendering. For use by the appliation to tell the control to change images on demand.</summary>
public void Render()
{
this.Invalidate();
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
public virtual void SetRenderFrameAndRender(Int32 key)
{
this.SetRenderFrameAndRender(key, false);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="freePreviousFrame">
/// Flag indicating whether to dispose of the previous image set
/// (in the case of transient images, such as composite images for a game or high-frame video playback)
/// </param>
public virtual void SetRenderFrameAndRender(Int32 key, Boolean freePreviousFrame)
{
this.SetRenderFrameAndRender(key, 0, 0, freePreviousFrame);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY)
{
this.SetRenderFrameAndRender(key, originX, originY, false);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
/// <param name="freePreviousFrame">
/// Flag indicating whether to dispose of the previous image set
/// </param>
public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY, Boolean freePreviousFrame)
{
lock (this.controlBufferLock)
{
Int32 previousFrameKey = this.currentFrameKey;
this.SetRenderFrame(key, originX, originY);
this.Render();
if (freePreviousFrame)
this.FreeFrameResource(previousFrameKey);
}
}
#endregion
}
}
Win32 Multimedia Timer Callback's Code
Excluded for size, but here is the relevant code, linking to "winmm.dll", EntryPoint = "timeSetEvent"
/// <summary>Callback method for WIn32 API</summary>
protected virtual void Win32Callback(UInt32 timerId, UInt32 message, IntPtr user, IntPtr param1, IntPtr param2)
{
TimeSpan raise;
lock (this.timerLock)
{
//get system time for start time
UInt32 uptime = Win32MultimediaTimeFunctions.GetEnvironmentUptime();
Int32 upTimeSec = (Int32)(uptime / 1000);
Int32 upTimeMilliSec = (Int32)(uptime % 1000);
TimeSpan WinUpTime = new TimeSpan(0, 0, 0, upTimeSec, upTimeMilliSec);
raise = WinUpTime - this.StartTime;
//this.elapsed(raise);
this.RaiseTimer(raise);
}
//Note: outside the lock, this.elapse(raise) kills the display driver
//inside, without threading, it is fine.
//inside, with threading, the display driver will crash after a bit of playback
}
protected void RaiseTimer(TimeSpan elapsedTime)
{
Thread timerEvent = new Thread(() => { this.RunThread(elapsedTime); });
timerEvent.IsBackground = true;
timerEvent.Name = "Timer callback";
timerEvent.Start();
}
/// <summary>Raises the timer on a separate thread</summary>
protected void RunThread(TimeSpan elapsedTime)
{
if (this.threadLock.WaitOne(0))
{
this.elapsed(elapsedTime);
this.threadLock.ReleaseMutex();
}
}
Conclusion
After 3 weeks, I'm not ashamed to say I'm baffled by this behavior, and looking for any sort of input for identifying a race condition, something that I might be missing, architecturally speaking, about Direc2D or where I'm just showing my ignorance.

All locks statements in the code seems to be an over complicated solution. Also creating COM objects (like Direct2D1) from constructors and using them from thread UI methods is usually not a good practice.
I would simplify your application by performing all device/objects creation and rendering on the same thread, also by keeping your rendering code outside the form. Ideally, this should be totally independent, as you could render to any kind of windows/controls via the HWND or to a DXGI surface, or to a Windows 8 Metro surface.
Also, as Aren suggest, instead of using your own Direct2D1 wrapper, you should rely on an existing robust wrapper like SharpDX. It will save you to maintain a costly layer and help you to focus on your specific application.