DISCLAIMER
This question is somewhat similar to another on StackOverflow, C# - Capturing the Mouse cursor image - but with a slightly different requirement.
BACKGROUND
I am writing a scriptable automation client that scraps data from 3 legacy Win32 systems.
Two of these systems may indicate the presence of finished tasks via a change in cursor bitmap when the cursor is hovered over some specific areas. No other hints (color change, status message) are offered.
My own code is derived from the original post mentioned on the disclaimer.
REQUIREMENTS
While I an able to capture the cursor bitmaps by programatically moving the cursor to a specific coordinate and capturing it via CURSORINFO, the idea was to allow an interactive user to continue using the computer. As it is, the forced positioning disrupts the process.
QUESTION
Is there a way to capture the cursor bitmap by parametrized position (e.g., request the CURSORINFO as if the focus was in window W at coordinates X, Y)?
A solution fulfilling the specifics of this question was implemented using the information provided by Hans Passant, so all credit must go to him.
The current setup is as shown:
It runs on a machine with two displays. Not shown in the picture is a small application that is actually responsible for the event monitoring and data scraping - it runs minimized and unattended.
Solution
Obtain the Window handle for the application to be tested (in this case, I cycled through all processes returned by Process.GetProcesses():
IntPtr _probeHwnd;
var _procs = Process.GetProcesses();
foreach (var item in _procs)
{
if (item.MainWindowTitle == "WinApp#1")
{
_probeHwnd= item.MainWindowHandle;
break;
}
}
With the window handle for the target application, we are now able to craft specific messages and send to it via SendMessage.
In order to pass coordinates to SendMessage we need to serialize both X and Y coordinates into a single long value:
public int MakeLong(short lowPart, short highPart)
{
return (int)(((ushort)lowPart) | (uint)(highPart << 16));
}
Knowing the specific coordinates we want to probe (_probeX,_probeY), now we can issue a WM_NCHITTEST message:
SendMessage(_probeHwnd, WM_NCHITTEST, NULL, (LPARAM)MakeLong(_probeX, _probeY));
We need GetCursorInfo to obtain the Bitmap:
Win32Stuff.CURSORINFO ci = new Win32Stuff.CURSORINFO();
Win32Stuff.GetCursorInfo(ci);
Check if the return flag from GetCursorInfo indicates that the cursor is showing (pco.flags == CURSOR_SHOWING):
Use CopyIcon in order to obtain a valid handle for the cursor bitmap:
IntPtr hicon = default(IntPtr);
hicon = Win32Stuff.CopyIcon(ci.hCursor);
Use GetIconInfo to extract the information from the handler:
Win32Stuff.ICONINFO icInfo = default(Win32Stuff.ICONINFO);
Win32Stuff.GetIconInfo(hicon, icInfo);
Use the System.Drawing.Icon class to obtain a manageable copy using Icon.FromHandle, passing the value returned by CopyIcon;
Icon ic = Icon.FromHandle(hicon);
Extract the bitmap via Icon.ToBitmap method.
Bitmap bmp = ic.ToBitmap();
Limitations
This solution was tested on two different OSes: Windows XP and Windows 8. It only worked on Windows XP. On Windows 8 the cursor would flicker and return to the 'correct' format immediately, and the the captured CURSORINFO reflected that.
The test point areas must be visible (i.e., application must not be minimized, and test points can't be under an overlapping window. Tested window may be partially overlapped, though - and it doesn't need to have focus.)
When WM_NCHITTEST is issued, the current physical cursor over WebApp changes to whatever cursor bitmap is set by the probed application. CURSORINFO contains the cursor bitmap set by the probed application, but the coordinates always indicate the 'physical' location.
Related
I have a DX11 Unity application, which is loading a native C++ DLL. The DLL creates its own D3D11 Device. I would like to take a texture from Unity and use it in my C++ DLL - ideally without copying to CPU memory.
On Unity side I do this :
MyNativeLib.SetBuffers11(srcTexture.GetNativeTexturePtr());
In this case, srcTexture is RenderTexture.
In the native DLL, I do this:
void SetBuffers11(ID3D11Resource* colorRes)
{
D3D11_TEXTURE2D_DESC texDesc;
ID3D11Texture2D* tempColor;
colorRes->QueryInterface(__uuidof(ID3D11Texture2D), (void**)&tempColor);
tempColor->GetDesc(&texDesc); //This works
ID3D11Resource* tempResource;
HRESULT openResult = m_device->OpenSharedResource1(
colorRes, __uuidof(ID3D11Resource), (void**)&tempResource); //This fails
tempResource->QueryInterface(__uuidof(ID3D11Texture2D), (void**)(&tempColor));
tempColor->GetDesc(&texDesc);
}
If I just query texture description, I get correct D3D11_TEXTURE2D_DESC. But when I try to access the texture data, I get MISCELLANEOUS CORRUPTION #18: CORRUPTED_PARAMETER. So I tried to use OpenSharedResource - It also failed but it told me, the pointer is probably NT (I am not actualy sure about that) and I should OpenSharedResource1. So I did and I get this :
D3D11 ERROR: ID3D11Device::OpenSharedResource1:
Returning E_INVALIDARG, meaning invalid parameters were passed.
[ STATE_CREATION ERROR #381: DEVICE_OPEN_SHARED_RESOURCE_INVALIDARG_RETURN]
OpenSharedResource1 returns E_INVALIDARG and pointer is set to 0. I am not sure which function to call for textures created by Unity in DX11 mode.
When I query for texture description, I get this:
Format : 0x09 (DXGI_FORMAT_R16G16B16A16_TYPELESS)
Bind Flags : 0x20 | 0x08 (D3D11_BIND_SHADER_RESOURCE and D3D11_BIND_RENDER_TARGET)
CPU ACCESS : 0
Any help appreciated
I finaly managed to get this working. I had several issues with my original approach.
To share a texture between two DX11 devices, the texture needs to have both D3D11_RESOURCE_MISC_SHARED_NTHANDLE and D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX set for MiscFlags. This a recommended approach since Windows 8. There are also some other restrictions when creating shareable texture, check the docs.
As far as I know, Unity wont let me specify these flags (maybe in a native plugin), but I can capture the original DX11 device, create a new shareable texture and copy the Unity texture to this new texture on the original Unity DX11 device.
ID3D11Device* otherDevice;
ID3D11Device1* otherDevice1;
unityTexture->GetDevice(&otherDevice);
otherDevice->QueryInterface(__uuidof (ID3D11Device1), (void**)&otherDevice1);
otherDevice->Release();// throw away the original device
otherDevice1->GetImmediateContext1(&otherContext);
//you can now create the shareable texture on otherContext
Notice that ID3D11Device1 is needed, since it contains methods for working with shared resources.
Now that you have shareable texture, you first need to share it on the original device, by creating a shared handle from the shareable texture :
IDXGIResource1* shareableResource;
//sharedTexture is the resource you want to share
sharedTexture->QueryInterface(__uuidof(IDXGIResource1),(void**)&shareableResource);
HANDLE sharedHandle;
HRESULT createSharedRes = shareableResource->CreateSharedHandle(NULL,
DXGI_SHARED_RESOURCE_READ,//other device will only READ
nullptr,
&sharedHandle);
shareableResource->Release();
This shared handle can now be passed to other DX11 device, which can open it :
m_device->OpenSharedResource1(sharedHandle, __uuidof(ID3D11Texture2D), (void**)sharedTextureLocal);
Now sharedTextureLocal contains a pointer to a ID3D11Texture2D on your local device, which you can use. However, when you want do move texture data from one device to another, you need to synchronize access to these resources, otherwise you will get garbage data. You need to synchronize access everytime time you access shared resources on both devices. In my case I needed to synchronize it twice, once on original Unity device when copying from Unity texture to shareable texture and then on my local device when using the local shared texture :
IDXGIKeyedMutex* keyedMutex;
sharedTexture->QueryInterface(__uuidof(IDXGIKeyedMutex), (void**)&keyedMutex);
keyedMutex->AcquireSync(0, INFINITE);
otherContext->CopyResource(dst, src);//use the correct context
keyedMutex->ReleaseSync(0);
In order to program an AI for a web video game, I'd like to get a screenshot of the game. No problem, i can use GetWindowRect.
But, this method actually save a screenshot without the filters applyed by the GPU. I mean, I'd like to get real colors of the webpage, not the one I see after the GPU processing.
The form recognition is actually based on colors and I can't publish this AI if nobody gets the same colors on the screenshot.
Is there any way to do that ?
--
PinkPR
You need to render in the same way as you render on screen but to bitmap's device context.
Sample code to provide basic idea on subject:
..
..
//Application of matrices and usual program flow
//
..
//Get graphics from the bitmap previously created
Graphics gc = Graphics.FromImage(bmp)
//Get device context
IntPtr hDc = gc.GetHdc();
int iPixelFormat = Gdi.ChoosePixelFormat(hDc, ref pfd);
Gdi.SetPixelFormat(hDc, iPixelFormat, ref pfd);
IntPtr hRc = Wgl.wglCreateContext(hDc);
//Make it current
if (!Wgl.wglMakeCurrent(hDc, hRc))
{
throw new Exception("....");
}
//Render to hDc
Using the answer in this question I can get the "screen" count. However, this doesn't seem to work with monitors that are set to "duplicate" (one monitor is reported instead of 2). My application prompts a user to switch from VGA to HDMI (this is on a device with both output ports), and then puts a "can you see this?" prompt on screen to verify that both video ports are working.
I am trying to detect that the switch has happened before showing the prompt, but due to the above mentioned problem the code does not see the monitor count decrement, then increment (that is how I am detecting the switch).
How can I detect the video device switch if everything is set to duplicate? The existing code works if the monitors are set to "extend". There is an internal video device that is always present as well (not trying to test this one).
See This question and use the provided (and fixed in the answer) wrapper for QueryDisplayConfig.
change the signature of the import to have out DisplayConfigTopologyId topology as the last parameter.
Use the QueryDisplayFlags.DatabaseCurrent for the display flags, otherwise you'll get status 87 (invalid parameter)
After calling QueryDisplayFlags the topology will be Clone, Extend etc.
call the method...
var status = CCDWrapper.QueryDisplayConfig(
CCDWrapper.QueryDisplayFlags.DatabaseCurrent,
ref numPathArrayElements, pathInfoArray, ref numModeInfoArrayElements,
modeInfoArray, out currentTopologyId);
In my tests numPathArrayElements always came back as the number of monitors currently In Use. If I changed it to "Show Only Screen 1", It said 1 screen, topology internal. "Show Only Screen 2" came back with 1 screen external. "Cloned" showed 2 screens.
James Barrass' answer didn't work for me. I ended up going with the answer here: link
Here's the code:
public static int GetScreenCount()
{
ManagementObjectSearcher searcher =
new ManagementObjectSearcher("root\\CIMV2",
"SELECT * FROM Win32_PnPEntity where service =\"monitor\"");
return searcher.Get().Count;
}
I'm trying to set up one of the above sources to capture the screen on a windows machine. Everything works fine apart from one important detail: I can't change the coordinates of the captured window in either src while pipeline is playing.
I'm currently using C# + Gstreamer-Sharp, yet I'm already considering a native C wrapper.
Should I be able to change the position of the window using the above plugins while playing?
If yes, could this be a problem related to Gstreamer-sharp?
Or does it require a procedure other than simply changing the values of the elements in the pipe?
This is what I do...
var pipeline = new Gst.Pipeline();
screenCapSource = Gst.ElementFactory.Make("gdiscreencapsrc", "ScreencapSource") as Gst.Base.PushSrc;
SetCapWindow(0,0,320,240); //<- works
//Capsfilter...
//X264 encoder
//udpsink
//link...
//launch pipleline
//working fine...
//When pipeline is playing, this doesn't work.
//I never change the dimensions, only the position of the window - to no effect.
public void SetCapWindow(Int32 x, Int32 y, Int32 width, Int32 height){
screenCapSource["x"] = x;
screenCapSource["y"] = y;
screenCapSource["width"] = width;
screenCapSource["height"] = height;
}
This issue has been posted before on the Gst mailing list by me and others but no replies.
BTW: A workaround would be using an Appsink to push screencaps into the pipeline from managed code, performance isn't great however.
I want to provide the user with a scaled-down screenshot of their desktop in my application.
Is there a way to take a screenshot of the current user's Windows desktop?
I'm writing in C#, but if there's a better solution in another language, I'm open to it.
To clarify, I need a screenshot of the Windows Desktop - that's the wallpaper and icons only; no applications or anything that's got focus.
You're looking for Graphics.CopyFromScreen. Create a new Bitmap of the right size and pass the Bitmap's Graphics object screen coordinates of the region to copy.
There's also an article describing how to programmatically take snapshots.
Response to edit: I misunderstood what you meant by "desktop". If you want to take a picture of the desktop, you'll have to:
Minimize all windows using the Win32API (send a MIN_ALL message) first
Take the snapshot
Then undo the minimize all (send a MIN_ALL_UNDO message).
A better way to do this would be not to disturb the other windows, but to copy the image directly from the desktop window. GetDesktopWindow in User32 will return a handle to the desktop. Once you have the window handle, get it's device context and copy the image to a new Bitmap.
There's an excellent example on CodeProject of how to copy the image from it. Look for the sample code about getting and creating the device context in the "Capturing the window content" section.
I get the impression that you are shooting for taking a picture of the actual desktop (with wallpaper and icons), and nothing else.
1) Call ToggleDesktop() in Shell32 using COM
2) Use Graphics.CopyFromScreen to copy the current desktop area
3) Call ToggleDesktop() to restore previous desktop state
Edit: Yes, calling MinimizeAll() is belligerent.
Here's an updated version that I whipped together:
/// <summary>
/// Minimizes all running applications and captures desktop as image
/// Note: Requires reference to "Microsoft Shell Controls and Automation"
/// </summary>
/// <returns>Image of desktop</returns>
private Image CaptureDesktopImage() {
//May want to play around with the delay.
TimeSpan ToggleDesktopDelay = new TimeSpan(0, 0, 0, 0, 150);
Shell32.ShellClass ShellReference = null;
Bitmap WorkingImage = null;
Graphics WorkingGraphics = null;
Rectangle TargetArea = Screen.PrimaryScreen.WorkingArea;
Image ReturnImage = null;
try
{
ShellReference = new Shell32.ShellClass();
ShellReference.ToggleDesktop();
System.Threading.Thread.Sleep(ToggleDesktopDelay);
WorkingImage = new Bitmap(TargetArea.Width,
TargetArea.Height);
WorkingGraphics = Graphics.FromImage(WorkingImage);
WorkingGraphics.CopyFromScreen(TargetArea.X, TargetArea.X, 0, 0, TargetArea.Size);
System.Threading.Thread.Sleep(ToggleDesktopDelay);
ShellReference.ToggleDesktop();
ReturnImage = (Image)WorkingImage.Clone();
}
catch
{
System.Diagnostics.Debugger.Break();
//...
}
finally
{
WorkingGraphics.Dispose();
WorkingImage.Dispose();
}
return ReturnImage;
}
Adjust to taste for multiple monitor scenarios (although it sounds like this should work just fine for your application).
You might look at using GetDesktopWindow through p/invoke, then get the device context and take your snapshot. There is a very detailed tutorial here. It might be better than perturbing the existing windows.
You can always catch screenshot on windows startup before applications run
or use other desktop by using desktop switching. Take a look here:
http://www.codeproject.com/Articles/7666/Desktop-Switching
Switch to other desktop, take picture and remember you dont need to show it just create it "DESKTOP"
All former answers are completely wrong (minimizing apps is absurd)
Simply use SHW api : one line of code (the desktop bitmap being cached, the api simply blits it to your HDC...)