I have a windows forms application where I would like to find the handle of a DirectX application, find a DirectX button in that window, and Click it.
Is this possible, and if so, where is the best place to start?
Thanks!
This is really not possible.
Unlike Windows Forms, a "DirectX Button" has no distinct windows handle. You don't actually "click" a button in a DirectX window, but rather send a mouse click to the entire DirectX window, and allow the application to do its own hit testing to "click" the "button" (which is just a graphic).
The closest you could do would be to send a mouse click to a location within the window - however, finding the location of a "button" could be very difficult.
Actually, it isn't too difficult if that is the case. A DX object can receive an click if you can isolate the object with the mouse through a reversal system, depending if it is 3D-based or 2D-based. 2D is using masking effects, and 3D adds on to translate to and from 2D/3D based space on certain properties utilized in vector calculus.
Pretty much from logical space, in 3D, a slice of the 3D object is mapped to 2D space, which is then mapped to camera space, which is then rendered. Crude, there is much more to it then that, but I'm keeping it as simple as it can be. Now the task is to find the 2D volume/per-pixel location, translate a copy of the 2D render of a 3D object on an image that will be/is already a DeviceContext for the monitor to show () to a map, and unionize a clickable object (which will run the intended code, just as a button should) to the 2D map object copy.
In other words, find where in the screen the object is rendered and generate a clickable mask over the rendered portion of the screen. Whether it is a bitmap mask, or a polygon mask, or a cube mask, or any other shape you can mathematically create; so long as the 2D clickable object actually interfaces with the mouse position and state in an constant-update system.
For example, say that application is full screen, that the resolution is 800 x 600 (keeping it simple, it could be anything), and that the object you wish to be able to interact, that is rendered in a context that doesn't permit it of it's own accord (*1), and that the rendered object shows a rectangle. Lets say that is the point of it, and the system you use for the translation outputs a Rectangle (C#.NET code, yea, but it's just an example, there are better floating around). So we can say that the mask rectangle coordinates are identical to to the rendered object coordinates.
So let's say: "Rectangle mask = new Rectangle(100, 100, 200, 100); // X starting point, Y starting point, Length, Width." Pretty self-explanatory. Add a per-frame cursor position compared with mask positions, will indicate if the cursor is above the mask, and with extra coding, can indicated if the cursor clicked (or other events) while being within the mask's range.
So let's say the button opens the main menu. And an event happens, particularly a "MouseClick" event, while the cursor hotspot is at Point(125, 200). Which of course was in the range of the mask. In this instance, through all that trivial code, opens the main menu. It is easy to add new clickable objects within this system if it is done right.
Yes, other, easier ways exist. But from them all, this is one of the more dynamic and flexible systems. Uses range from the cash you get as loot in a game that you have to click to pick up, to the UI, to the menu's, to anything clickable. The range isn't in just games, either. Custom UI designers who like to reinvent the wheel can and have taken advantage of similar systems, such as myself, like to define every aspect from the microcode to the macrocode. Precisely, to define the macrocode as it is needed, as it is expected and without unnecessary bloat through microcode management. Not saying I like microcode, as it can be way too bloody finicky, and often times, is. Worth it for the few hundred FPS you get from streamlining it right (the first time), though... >:-D
*1 = (predictable in 3D Rendering models, from the point of view Rendering is not Modifying (which seems to be including object selection (through the mouse/cursor position and all that), which is essentially what your looking for) and as such not included (directly). Through translation it becomes possible, with the finest precision if done right. Further to that, DirectX, OpenGL, and others like, are defaultly Renderers.)
Related
I have no experience with images. I have to detect simple object in static image. For example I have image like:
I want to detect edges and remove background. Just to compare them.
Something like this.
Do u have any solutions of this problem? Images have often white backgrounds.
I've just thought about detect edges, and take everything what they contains.
To segment out the shoe-
Anadptive Threshold to remove the smooth changing background.
Sobelx, which removes the apparent background line, which i assume is
common for images of this setup.
dilate, closing operation to separate out the shoe.
Find contours, bounding box etc as per your choice.
Do an additional threshold if you want to remove the shadow at the bottom.
I'm working on an experimental project in which the challenge is to identify and extract an image of the icon or control that the user is has clicked on/touched. The method I'm trying is as follows (I need some help with step 3):
1) Take a screen shot when the user clicks/touches the screen:
2) Apply edge detection:
3) Extract the possible icon images around the Point associated with the user's cursor (Don't know how to do this)
There are easier cases in which the mouse-over event will highlight the icon/control, which allows me to identify the control with a simple screen shot comparison (before and after mouse-over). The above method is specifically for cases in which the icon is not highlighted. I'm new to emgu, so if anyone has any pointers on how to better achieve this, I'm all ears.
Cheers! Matt
Instead of doing edge detection. Consider taking the following steps:
Only grab pixels which are within a certain radius of the point of the user's cursor. Create a new image with just these pixels.
Use thresholding to classify into foreground and background.
Calculate the centroid, (use mean x coordinate and mean y coordinate). Calculate deviation from the mean. Discard foreground pixels which are beyond a certain deviation from the mean. Eg: discard pixels that are more than 1.6 deviations from the mean.
(You may need to experiment with this step ).
Use a convex hull to find the area of the image with the icon in it.
I am creating a custom control for semiconductor wafermap
Each of those small rectangle need to satisfy following requirements;
1) Tooltip to show the index
2) clickable to include or exclude from the wafermap definition.
no of dies in the wafermap may cross 2 millions in the case of 1400 x 1450 dies.
at certain point i need to show all the dies in a window (most of the clicking will happen in zoomed view).
Currently I am adding each die separately using Rectangle shape and store the coordinate information (index like (10,10)) for the tooltip as an attached property.
I use different style to each die; depending on certain calculation and position of the die.
DieStyle1 = new Style { TargetType = typeof(Rectangle) };
DieStyle1.Setters.Add(new Setter(Shape.FillProperty, Brushes.MediumSlateBlue));
DieStyle1.Setters.Add(new Setter(Shape.StrokeProperty, Brushes.White));
DieStyle1.Setters.Add(new EventSetter(MouseDownEvent, new MouseButtonEventHandler(DieStyle1_MouseDown)));
this approach is slow and use high memory too. so suggest a better way to achieve this in WPF?
In creating a designer for christmas tree lights, I ran into the same problem. Using UIElement or Shapes is way too slow when you get to 100+ items. The best approach to handle a very large number of items entails using double-buffering with your own managed buffer of the image and a structure to handle the clicks. I have posted my project which should give you a good start. It can be obtained at:
http://sourceforge.net/projects/xlightsdesigner/
You are interested in the Controls\ChannelitemsCanvas.cs. It can be modified to suit your needs and uses a quad-tree to store the rectangles so that click events can be quickly determined.
I want to create a filter over a specific area of the screen to perform filtering opertions.
Examples what a filtering opertion might be:
- inverting (e.g. change black pixel to white pixels, red to cyan)
- masking pixels (e.g. mask = ff0000; input c79001 -> c70000)
- operations like photoshop's layer effects
Here is an example of what it should look like:
http://img443.imageshack.us/img443/1462/overlayk.png
Does anyone know how to perform this under Windows OS.
(my prefered language is C#)
Thanks!
Depending on how fast you need the "filter" to update, a quick and hacky way is just to get a screenshot using CopyFromScreen while your filter window is invisible, apply the filter to the image data, and then set the filter window to display the image data.
If you want to do it without having to hide the window first, you'll probably need to do something like http://www.codeproject.com/KB/system/snapshot.aspx where you capture individual windows.
An even trickier but potentially faster thing to do, and requiring nearly complete use of p/invoke win32 calls, would be to not have a window at all, get the required capture windows based on their coordinates, capture the images as above, and then draw directly to the screen DC.
To clarify: you want an area of the desktop, not just within the bounds of your window, to be under your control allowing you to apply a per-pixel filter. If that's the case, I think what you need is DirectDraw using the XNA libraries. WPF MAY get you what you need, but WinForms will most likely not. There are third party tools as well.
If you want this capability only within the bounds of your application's window, for instance in a drawing application, it gets far easier. Every class in the Windows.Forms namespace that inherits from Control exposes a CreateGraphics() method. This method returns an object representing a drawing surface covering the screen area of the control, and is the basis for just about anything you want to do on a window involving custom graphics (and internally, it's used to draw the controls in the first place).
Once you have the Graphics object, you can draw Images on it. A popular method of drawing custom graphics like animations or games is to do the actual drawing on a Bitmap object (derived from the abstract Image) and then when you're done, draw the Bitmap on the Graphics area. This is done to reduce flicker; if the graphics area is shown to the user while it is being drawn on, the user will only see the complete image for a split second before it is "wiped" and redrawn, and shapes drawn halfway through will be there one moment and gone the next as they wait to be drawn. Drawing to a bitmap, then showing the Bitmap on the screen when you're done, means the user sees a complete image at a time.
You can extend this using transparency features to create multi-layered images. Have a Bitmap for every layer you wish to manipulate. Work on them seperately, then draw each of them, in their proper order from back to front, onto a master Bitmap, and draw that Bitmap on the screen. This allows you those PhotoShop-type manipulations where a part of the image is one layer, and can be manipulated independently of all others.
As for per-pixel filtering, Bitmap objects expose GetPixel() and SetPixel() methods, which allow you to grab the color of a single pixel, perform a filter calculation, and re-draw it. This process will be totally unaccelerated, and so limited by your CPU speed, but allow very fine control of your image, or repetitive tasks like your filters.
I would like to develop a simple point&click game in C# with the default drawing libraries...no openGL/SDL/Tao for this project.
Suffice to say, I am curious as to the best ways to draw clickable images in layers on a form.
Ideally, I would have
1) Environment layer (pathways, doors, etc)
2) Object layer (items)
3) Character layer (enemies)
Ideally, the layers beneath other layers would still be viewable, so I could still see the environment underneath an object (so whatever component I use to draw the object to the form needs to be transparent).
This game is going to be tile based...so I will be generating a 2D array of some sort of component and putting those onto the form. The question is, what component should I use? A friend has recommended to generate panels and drop those onto the form and use the background image property, but is there a better way?
I know that this is not the ideal way to develop...this is more of a prototype for myself. Later on I will probably move it to Tao if I get anywhere, but for now (ie, the next year or so), I would like to keep it extremely simple.
Tiles are simply logical chunks with which to organize your UI. You do not need to necessarily have tiles in the form of Panels to use the "tile" idea. Switching tiles can simply mean that your character has reached the right edge of tile A and you will therefore draw tile B and place the character on the left edge of that tile. So your memory structure should be a 2D array of logical tile objects that contain information about what to draw but not necessarily Panels, though that is an option, i don't recommend it.
Layers: You don't really need to worry about layers. All you have to do is draw in the correct sequence: Environment first, followed by objects and then characters. .Net and probably all frameworks paint back to front, which means that things drawn first may be entirely or partially obscured by objects drawn later. The "layering" happens automatically. If you just draw your objects within their own bounds then you do not need to worry about transparency. E.g. a 20x20 pixel character is drawn within a 20x20 pixel bound.
This is all assuming raw e.Graphics.Draw* and e.Graphics.Fill* calls in the Paint event handler of course. As with anything in software, there are 101 ways...