Drawing overlays with GDI on video that is rendered with directx - c#

I am using ffmpeg, hardware acceleration and directx 9 for video stream application. The application includes c# for user interface and c++ for rendering. At the beginning, I used directx for simple overlays like line, rectangle, polygon, triangle, text etc. But now, I need more complex overlays. Then, I think to use directx for video and GDI for overlays.
I tried to call GDI fuction in c# side for the same handle when directx Present is finished. Actually it works but overlay, that is drawn by GDI, is flickering.
I changed PresentationInterval in D3DPRESENT_PARAMETERS to D3DPRESENT_INTERVAL_TWO and D3DPRESENT_INTERVAL_THREE. Result is application crashed.
I called LockWindowUpdate(myHandle) before directx Present and called LockWindowUpdate(IntPtr.Zero) after GDI function. Result is application crashed.
1.Is drawing overlay with GDI on directx a nice solution?
If YES, is there a way to do it without flickering?
If NO, should i do all work with directx?
Update: Panel control , that displays video, is already set DoubleBuffered
System.Reflection.PropertyInfo prop =
typeof(System.Windows.Forms.Control).GetProperty(
"DoubleBuffered",
System.Reflection.BindingFlags.NonPublic |
System.Reflection.BindingFlags.Instance);
prop.SetValue(displayPnl,true,null);

Related

RichTextBox: Transparent Background vs. Font Quality in Bitmap

I created a RTF control pretty similar to the one described here.
Works pretty fine. Sometimes i have the control draw its contents directly to the device context, sometimes i use an offscreen bitmap which i draw to the device context myself, depending on the situation.
Now i needed the control to have a transparent background so i've overriden the CreateParams method called on initalising the base clss and added createParams.ExStyle |= 0x20; which boils down to one of the extended window styles, WS_EX_TRANSPARENT.
At this point, transparency works like a charm. But in contrast to before, setting this option makes drawing to the offscreen bitmap unusable, since the font quality suffers. Note, that only when drawing to the bitmap the font quality turns bad, drawing to the device context of the control is still working although creating the offscreen bitmap uses the same routines, just with a different device context.
As you can see, the font on the left is perfectly fine, while on the right side, the font get messy, too bold and it seems like it's drawn over several times with the coordinates not matching.
Commenting out the line added for transparency makes the bitmap return to normal but also yields in a white background for everything drawn, making me loose transparency.
Any ideas how to get a successfull combination of the two desired effects?

rectangle drawn on desktop is not persistent in c# [duplicate]

I want to draw directly on the desktop in C#. From searching a bit, I ended up using a Graphics object from the Desktop HDC (null). Then, I painted normally using this Graphics object.
The problem is that my shapes get lost when any part of the screen is redrawn. I tried a While loop, but it actually ends up drawing as fast as the application can, which is not the update rate of the desktop.
Normally, I would need to put my drawing code in a "OnPaint" event, but such thing does not exist for the desktop.
How would I do it?
Example code: https://stackoverflow.com/questions/1536141/how-to-draw-directly-on-the-windows-desktop-c
I posted two solutions for a similar requirement here
Basically you have two options.
1- Get a graphics object for the desktop and start drawing to it. The problem is if you need to start clearing what you have previously drawn etc.
Point pt = Cursor.Position; // Get the mouse cursor in screen coordinates
using (Graphics g = Graphics.FromHwnd(IntPtr.Zero))
{
g.DrawEllipse(Pens.Black, pt.X - 10, pt.Y - 10, 20, 20);
}
2- The second option that I provide in the link above is to create a transparent top-most window and do all your drawing in that window. This basically provides a transparent overlay for the desktop which you can draw on. One possible downside to this, as I mention in the original answer, is that other windows which are also top-most and are created after your app starts will obscure your top most window. This can be solved if it is a problem though.
For option 2, making the form transparent is as simple as using a transparency key, this allows mouse clicks etc. to fall through to the underlying desktop.
BackColor = Color.LightGreen;
TransparencyKey = Color.LightGreen;
When you draw to HDC(NULL) you draw to the screen, in an unmanaged way. As you've discovered, as soon as windows refreshes that part of the screen, your changes are overwritten.
There are a couple of options, depending upon what you want to achieve:
create a borderless, possibly
non-rectangular window. (Use
SetWindowRgn to make a window
non-rectangular.) You can make this a child of the desktop window.
subclass the desktop window. This is not straightforward, and involves
injecting a DLL into the
Explorer.exe process.
To get an OnPaint for the desktop you would need to subclass the desktop window and use your drawing logic when it receives a WM_PAINT/WM_ERASEBKGND message.
As the thread you linked to says, you can only intercept messages sent to a window of an external process using a hook (SetWindowsHookEx from a DLL).
As mentioned a transparent window is another way to do it, or (depending on the update frequency) copying, drawing and setting a temporary wallpaper (as bginfo does).
This is difficult to do correctly.
It will be far easier, and more reliable, to make your own borderless form instead.

screen color filtering

I want to create a filter over a specific area of the screen to perform filtering opertions.
Examples what a filtering opertion might be:
    - inverting (e.g. change black pixel to white pixels, red to cyan)
    - masking pixels (e.g. mask = ff0000; input c79001 -> c70000)
    - operations like photoshop's layer effects
Here is an example of what it should look like:
http://img443.imageshack.us/img443/1462/overlayk.png
Does anyone know how to perform this under Windows OS.
(my prefered language is C#)
Thanks!
Depending on how fast you need the "filter" to update, a quick and hacky way is just to get a screenshot using CopyFromScreen while your filter window is invisible, apply the filter to the image data, and then set the filter window to display the image data.
If you want to do it without having to hide the window first, you'll probably need to do something like http://www.codeproject.com/KB/system/snapshot.aspx where you capture individual windows.
An even trickier but potentially faster thing to do, and requiring nearly complete use of p/invoke win32 calls, would be to not have a window at all, get the required capture windows based on their coordinates, capture the images as above, and then draw directly to the screen DC.
To clarify: you want an area of the desktop, not just within the bounds of your window, to be under your control allowing you to apply a per-pixel filter. If that's the case, I think what you need is DirectDraw using the XNA libraries. WPF MAY get you what you need, but WinForms will most likely not. There are third party tools as well.
If you want this capability only within the bounds of your application's window, for instance in a drawing application, it gets far easier. Every class in the Windows.Forms namespace that inherits from Control exposes a CreateGraphics() method. This method returns an object representing a drawing surface covering the screen area of the control, and is the basis for just about anything you want to do on a window involving custom graphics (and internally, it's used to draw the controls in the first place).
Once you have the Graphics object, you can draw Images on it. A popular method of drawing custom graphics like animations or games is to do the actual drawing on a Bitmap object (derived from the abstract Image) and then when you're done, draw the Bitmap on the Graphics area. This is done to reduce flicker; if the graphics area is shown to the user while it is being drawn on, the user will only see the complete image for a split second before it is "wiped" and redrawn, and shapes drawn halfway through will be there one moment and gone the next as they wait to be drawn. Drawing to a bitmap, then showing the Bitmap on the screen when you're done, means the user sees a complete image at a time.
You can extend this using transparency features to create multi-layered images. Have a Bitmap for every layer you wish to manipulate. Work on them seperately, then draw each of them, in their proper order from back to front, onto a master Bitmap, and draw that Bitmap on the screen. This allows you those PhotoShop-type manipulations where a part of the image is one layer, and can be manipulated independently of all others.
As for per-pixel filtering, Bitmap objects expose GetPixel() and SetPixel() methods, which allow you to grab the color of a single pixel, perform a filter calculation, and re-draw it. This process will be totally unaccelerated, and so limited by your CPU speed, but allow very fine control of your image, or repetitive tasks like your filters.

Taking screenshots in Windows Vista, Windows 7, with transparent areas outside the app region

I am trying to take a screenshot of an application and I would like to make the parts of the rectangle that are not part of the applications region be transparent. So for instance on a standard windows application I would like to make the rounded corners transparent.
I wrote a quick test application which works on on XP (or vista/windows 7 with aero turned off):
protected override void OnPaint(PaintEventArgs e)
{
base.OnPaint(e);
Graphics g = e.Graphics;
// Just find a window to test with
IntPtr hwnd = FindWindowByCaption(IntPtr.Zero, "Calculator");
WINDOWINFO info = new WINDOWINFO();
info.cbSize = (uint)Marshal.SizeOf(info);
GetWindowInfo(hwnd, ref info);
Rectangle r = Rectangle.FromLTRB(info.rcWindow.Left, info.rcWindow.Top, info.rcWindow.Right, info.rcWindow.Bottom);
IntPtr hrgn = CreateRectRgn(info.rcWindow.Left, info.rcWindow.Top, info.rcWindow.Right, info.rcWindow.Bottom);
GetWindowRgn(hwnd, hrgn);
// fill a rectangle which would be where I would probably
// write some mask color
g.FillRectangle(Brushes.Red, r);
// fill the region over the top, all I am trying to do here
// is show the contrast between the applications region and
// the rectangle that the region would be placed in
Region region = Region.FromHrgn(hrgn);
region.Translate(info.rcWindow.Left, info.rcWindow.Top);
g.FillRegion(Brushes.Blue, region);
}
When I run this test app on XP (or Vista/Windows 7 with Aero off), I get something like this, which is great because I can eek an xor mask out of this that can be used later with BitBlt.
removed dead Imageshack link - Screenshot
Here is the problem, on Vista or Windows 7 with Aero enabled, there isn't necessarily a region on the window, in fact in most cases there isn't. Can anybody help me figure out some way to build a mask like this on these platforms?
Here are some of the approaches I have already tried...
1. Using the PrintWindow function: This doesn't work because it gives back a screenshot taken of the window with Aero off and this window is a different shape from the window returned with Aero on
2 Using the Desktop Window Manager API to get a full size thumbnail: This didn't work because it draws directly to the screen and from what I can tell you can't get a screenshot directly out of this api. Yeah, I could open a window with a pink background, show the thumbnail, take a screenshot then hide this temporary window but thats a horrible user experience and a complete hack I would rather not have my name on.
3. Using Graphics.CopyFromScreen or some other pinvoke variant of this: This doesn't work because I can't assume that the window I need information from is at the top of the z-order on the screen.
Right now, the best solution I can think of is to special case Aero on Windows 7 and Vista to manually rub out the corners by hard coding some graphics paths I paint out but this solution would suck since any application that performs custom skinning will break this.
Can you think of another or better solution?
If you are here, thanks for taking time to read this post, I appreciate any help or direction that you can offer!
If you are looking for a finished application, there is 7capture, which captures also the translucency, so images can be saved to PNG format for later compositing.
EDIT:
The original question and comments indicate you are looking to produce a region on Windows Vista/7 that you can then use to mask out parts the captured image, as is done with Windows XP and non-Aero UIs. Using a region is not going to give you the result you are looking for, since the window outline is not computed as a region, but as an image with variable transparency - RGBA. The Alpha channel in that image is your mask, but it's not an on-off mask like a region, but a gradual mask with a range of values from pixels being fully included to being fully masked out.
Although it uses undocumented APIs, the code at http://spazzarama.wordpress.com/2009/02/12/screen-capture-with-vista-dwm/ will capture to a RGBA buffer which you can then use to render or save the image with the shadow and other translucency effects intact.
In DwmCapture.cs Change
BackBufferFormat = Format.X8R8G8B8
to
BackBufferFormat = Format.A8R8G8B8
(X8->A8)
And you should then be able to access both the usual RGB data plus transparency from the captured buffer. This can then be saved as a PNG or other format with alpha-channel for composing.
Removed idea that is terrible but would have been awesome back in the '90s
You say that using the DWM API only allows you to capture directly to the screen... could you create a window offscreen (say, X = -100000px, Y = -100000px) but visible (maybe even hidden?) and draw the screenshot to it? Since when using the DWM each window has a backing texture, I'm thinking it might still get drawn fine even though the target isn't directly onscreen.
Also, if you want to go the DirectX route and access the actual DX texture backing the window, I found a few leads that might help (especially the first link):
http://spazzarama.wordpress.com/2009/02/12/screen-capture-with-vista-dwm/
http://channel9.msdn.com/forums/TechOff/251261-Help-Getting-the-shared-window-texture-out-of-DWM-/
http://web.archive.org/web/20080625183653/http://www.aeroxp.org/board/index.php?showtopic=6286
http://www.youtube.com/watch?v=hRTgFTMnT_U
Using Graphics.CopyFromScreen or some other pinvoke variant of this:
This doesn't work because I can't
assume that the window I need
information from is at the top of the
z-order on the screen.
If you know which window you need the information from, can you bring it to the front, call Graphics.CopyFromScreen, and then reset its z-index? I know from experience that Aero does odd things when items are in the background in order to make their glass interface work correctly (partial rendering etc). This may not be great UX; however, it would be a special case and used only when Aero is turned on.
You can take a look at the source code of AeroShot, as described on the main page, it can capture rounded edges and with the Aero Glass transparency effect and save it to a PNG file. It's written in C#.

Resizing window causes black strips

I have a form, which sets these styles in constructor:
this.SetStyle(ControlStyles.AllPaintingInWmPaint, true);
this.SetStyle(ControlStyles.UserPaint, true);
this.SetStyle(ControlStyles.ResizeRedraw, true);
this.SetStyle(ControlStyles.OptimizedDoubleBuffer, true);
And I draw some rectangles in Paint event. There are no controls on the form. Hovewer, when I resize the form, there are black strips at right and bottom of the form. Is there any way to get rid of them? I've tried everything, listening for WM_ERASEBKGND in WndProc, manually drawing the form on WM_PAINT, implementing custom double buffer, etc. Is there anything else I could try?
I've found this:
https://connect.microsoft.com/VisualStudio/feedback/details/522441/custom-resizing-of-system-windows-window-flickers
and it looks like it is a bug in DWM, but I just hope I can do some workaround.
Please note that I must use double buffering, since I want to draw pretty intense graphic presentation in the Paint event. I develop in C# .NET 2.0, Win7.
Status Update 1
I've managed to get rid of most of the black stripes by implementing the resize functionality by myself. Hovewer there are still some minor glitches. Is there any way to do resize and paint operation at once? Here is a pseudo-code of what I need to do:
IntPtr hDC;
var size = new Size(250, 200);
IntPtr handle = API.PaintAndResizeBegin(this.Handle /* Form.Handle */,
size.Width, size.Height, out hDC);
using (var g = Graphics.FromHdc(hDC)) {
this.backBuffer.Render(g, size);
}
API.PaintAndResizeCommit(handle);
Is there any way to implement the above code?
The second solution could be to back-buffer whole form, including non-client area. But how to do that? I don't want to paint the non-client area by myself, as I want to keep the nice aero effect on Vista/7. Any help will be deeply appreciated.
Status Update 2
It looks like this problem is unsolvable, since it is omnipresent on Windows, in every application. We can just hope that MS will take some inspiration in Mac OS X and will provide appropriate APIs in new Windows.
I've found the function which can paint and resize window at the same time - UpdateLayeredWindow.
So now it should be possible to create resizable windows, which do not have any strips while being resized. However, you need to paint the window content yourself, so it is a little inconvenient. But I think that using WPF and UpdateLayeredWindow, there shouldn't be any problem.
Update
Found problems. :-) When using UpdateLayeredWindow, you must paint the window's border yourself. So, if you want standard window painted using UpdateLayeredWindow with nice glass effect in win7, you are screwed.
On Microsft Connect is even a thread about this problem, where Microsoft says it is a bug by design, and if it ever gets fixed, then probably in Win8 or some newer system. So there isn't much we could do about this.
I found that it is best not to do any custom rendering directly on the Form surface. Instead, put a docked PictureBox on the form, create Bitmap object that will be displayed in the PictureBox, draw everything onto that using the System.Drawing.Graphics.FromImage(Image) method.
I used that method with a game loop to make a simple shooter game (Crimsonland-style) and got pretty good performance (with anti-aliased lines), above 100 FPS.

Categories