We Keep Coding

Unity: screen space vs window space vs monitor space?

I am trying to take a Ui object's screen space position and translate that to what I am calling 'monitor space'.
As far as I can tell, screen space, in Unity, is relative to the applications' window. That is, even if the app is not full screen, and moved around on your monitor, 0,0 will still be the lower left of the app window.
I need to translate one of those screen space values into the actual position within the user's monitor. This is especially important when considering that the user might have multiple monitors.
I am not finding anything to get this done, though.
I am hoping to find a platform agnostic solution, but if it must be Windows-only than I can make that work as well.
Any help on this would be greatly appreciated.
Thank you

Now after TEEBQNE's answer I also wanted to give it a shot using the native solution.
As mentioned this will be only for Windows PC Standalone and requires
Unity's new Input System (see Quick Start)
One of the solutions from Getting mouse position in c#
For example if you want to use System.Windows.Forms then copy the according DLL from
C:\Windows\Microsoft.NET\Framework64\v4.x.xx
into your project under Assets/Plugins
Then in code you can use
using System.Windows.Forms;
If this is more efficient (or even works this way) I can't tell - only on the phone here - but I hope the idea gets clear ;)
So the idea is:
store initial cursor position
Set your cursor to certain positions of interest using WarpCursorPosition using Unity screen coordinates as input
read out the resulting absolute monitor coordinates using the native stuff
in the end reset the cursor to the original position
This might look somewhat like
using UnityEngine;
using UnityEngine.InputSystem;
public static class MonitorUtils
{
// Store reference to main Camera (Camera.main is expensive)
private static Camera _mainCamera;
// persistent array to fetch rect corners
// cheaper then everytime creating and throwing away a new array
// especially when fetching them every frame
private static readonly Vector3[] corners = new Vector3[4];
// For getting the UI rect corners in Monitor pixel coordinates
public static void GetMonitorRectCorners(this RectTransform rectTransform, Vector2Int[] output, bool isScreenSpaceCanvas = true, Camera camera = null)
{
// Lazy initialization of optional parameter
if (!camera) camera = GetMainCamera();
// Store initial mouse position
var originalMousePosition = Mouse.current.position.ReadValue();
// Get the four world space positions of your RectTtansform's corners
// in the order bottom left, top left, top right, bottom right
// See https://docs.unity3d.com/ScriptReference/RectTransform.GetWorldCorners.html
rectTransform.GetWorldCorners(corners);
// Iterate the four corners
for (var i = 0; i < 4; i++)
{
if (!isScreenSpaceCanvas)
{
// Get the monitor position from the world position (see below)
output[i] = WorldToMonitorPoint(corners[i], camera);
}
else
{
// Get the monitor position from the screen position (see below)
output[i] = ScreenToMonitorPoint(corners[i], camera);
}
}
// Restore mouse position
Mouse.current.WarpCursorPosition(originalMousePosition);
}
// For getting a single Unity world space position in Monitor pixel coordinates
public static Vector2Int WorldToMonitorPoint(Vector3 worldPoint, Camera camera = null)
{
// Lazy initialization of optional parameter
if (!camera) camera = GetMainCamera();
var screenPos = camera.WorldToScreenPoint(worldPoint);
return ScreenToMonitorPoint(screenPos, camera);
}
// For getting a single Unity world space position in Monitor pixel coordinates
public static Vector2Int ScreenToMonitorPoint(Vector3 screenPos, Camera camera = null)
{
// Lazy initialization of optional parameter
if (!camera) camera = GetMainCamera();
// Set the system cursor position there based on Unity screen space
Mouse.current.WarpCursorPosition(screenPos);
// Then get the actual system mouse position (see below)
return GetSystemMousePosition();
}
// Get and store the main camera
private static Camera GetMainCamera()
{
if (!_mainCamera) _mainCamera = Camera.main;
return _mainCamera;
}
// Convert the system mouse position to Vector2Int for working
// with it in Unity
private static Vector2Int GetSystemMousePosition()
{
var point = System.Windows.Forms.Cursor.Position;
return new Vector2Int(point.X, point.Y);
}
}
So you can either simply use
var monitorPosition = MonitorUtils.WorldToMonitorPoint(someUnityWorldPosition);
// or if you already have the `Camera` reference
//var monitorPosition = MonitorUtils.WorldToMonitorPoint(someUnityWorldPosition, someCamera);
or if you already have a screen space position like e.g. in a ScreenSpace Overlay canvas
var monitorPosition = MonitorUtils.ScreenToMonitorPoint(someUnityWorldPosition);
// or if you already have the `Camera` reference
//var monitorPosition = MonitorUtils.ScreenToMonitorPoint(someUnityWorldPosition, someCamera);
or you can get all four corners of a UI element at once using e.g.
var monitorCorners = new Vector2Int [4];
someRectTransform.GetMonitorRectCorners(monitorCorners, isScreenSpaceCanvas);
// or again if you already have a camera reference
//someRectTransform.GetMonitorRectCorners(monitorCorners, isScreenSpaceCanvas, someCamera);
Little example
public class Example : MonoBehaviour
{
[Header("References")]
[SerializeField] private Camera mainCamera;
[SerializeField] private RectTransform _rectTransform;
[SerializeField] private Canvas _canvas;
[Header("Debugging")]
[SerializeField] private bool isScreenSpace;
[Header("Output")]
[SerializeField] private Vector2Int bottomLeft;
[SerializeField] private Vector2Int topLeft;
[SerializeField] private Vector2Int topRight;
[SerializeField] private Vector2Int bottomRight;
private readonly Vector2Int[] _monitorPixelCornerCoordinates = new Vector2Int[4];
private void Awake()
{
if (!mainCamera) mainCamera = Camera.main;
if (!_canvas) _canvas = GetComponentInParent<Canvas>();
isScreenSpace = _canvas.renderMode == RenderMode.ScreenSpaceOverlay;
}
private void Update()
{
if (Keyboard.current.spaceKey.isPressed)
{
_rectTransform.GetMonitorRectCorners(_monitorPixelCornerCoordinates, isScreenSpace);
bottomLeft = _monitorPixelCornerCoordinates[0];
topLeft = _monitorPixelCornerCoordinates[1];
topRight = _monitorPixelCornerCoordinates[2];
bottomRight = _monitorPixelCornerCoordinates[3];
}
}
}
You will see that moving your mouse each and every frame isn't a good idea though ^^
Now you can see the four corners being updated depending on the actual position on the screen.
Note: while Unity Screenspace is 0,0 at the bottom left in normal display pixels 0,0 is actually rather top-left. So you might need to invert these.

Alright first off - sorry for the late response just got back and was able to type up an answer.
From what I have found, this solution does not work in the editor and produces odd results on Mac with retina display. In the editor, the Screen and Display spaces appear to be exactly the same. There is probably a solution to fix this but I did not look into the specifics. As for Mac, for whatever reason, the internal resolution outputted is always half the actual resolution. I am not sure if this is just a retina display bug with Unity or a general Mac bug. I tested and ran this test script on both a Windows computer and Mac with a retina display. I have yet to test it on any mobile platform.
I do not know exactly what you would like to achieve with the values you wish to find, so I set up a demo scene displays the values instead of using them.
Here is the demo script:
using UnityEngine;
using System.Collections.Generic;
using UnityEngine.UI;
public class TestScript : MonoBehaviour
{
[SerializeField] private RectTransform rect = null;
[SerializeField] private List<Text> text = new List<Text>();
[SerializeField] private Canvas parentCanvas = null;
[SerializeField] private Camera mainCam = null;
private void Start()
{
// determine the canvas mode of our UI object
if (parentCanvas == null)
parentCanvas = GetComponentInParent<Canvas>();
// only need a camera in the case of camera space canvas
if (parentCanvas.renderMode == RenderMode.ScreenSpaceCamera && mainCam == null)
mainCam = Camera.main;
// generate initial data points
GenerateData();
}
/// <summary>
/// Onclick of our button to test generating data when the object moves
/// </summary>
public void GenerateData()
{
// the anchored position is relative to screen space if the canvas is an overlay - if not, it will need to be converted to screen space based on our camera
Vector3 screenPos = parentCanvas.renderMode == RenderMode.ScreenSpaceCamera ? mainCam.WorldToScreenPoint(transform.position) : rect.transform.position;
// our object relative to screen position
text[0].text = "Screen Pos: " + screenPos;
// the dimensions of our screen (The current window that is rendering our game)
text[1].text = "Screen dimensions: " + Screen.width + " " + Screen.height;
// find our width / height normalized relative to the screen space dimensions
float x = Mathf.Clamp01(screenPos.x / Screen.width);
float y = Mathf.Clamp01(screenPos.y / Screen.height);
// our normalized screen positions
text[2].text = "Normalized Screen Pos: " + x + " " + y;
// grab the dimensions of the main renderer - the current monitor our game is rendered on
#if UNITY_STANDALONE_OSX
text[3].text = "Display dimensions: " + (Display.main.systemWidth * 2f) + " " + (Display.main.systemHeight * 2f);
// now find the coordinates our the UI object transcribed from screen space normalized coordinates to our monitor / resolution coordinates
text[4].text = "Display relative pos: " + (Display.main.systemWidth * x * 2f) + " " + (Display.main.systemHeight * y * 2f);
#else
text[3].text = "Display dimensions: " + Display.main.systemWidth + " " + Display.main.systemHeight;
// now find the coordinates our the UI object transcribed from screen space normalized coordinates to our monitor / resolution coordinates
text[4].text = "Display relative pos: " + (Display.main.systemWidth * x) + " " + (Display.main.systemHeight * y);
#endif
}
/// <summary>
/// Just for debugging - can be deleted
/// </summary>
private void Update()
{
if (Input.GetKey(KeyCode.A))
{
rect.anchoredPosition += new Vector2(-10f, 0f);
}
if (Input.GetKey(KeyCode.W))
{
rect.anchoredPosition += new Vector2(0f, 10f);
}
if (Input.GetKey(KeyCode.S))
{
rect.anchoredPosition += new Vector2(0f, -10f);
}
if (Input.GetKey(KeyCode.D))
{
rect.anchoredPosition += new Vector2(10f, 0f);
}
}
}
I accounted for the parent canvas being either Overlay or Camera mode and put in a check for an OSX build to adjust to the proper screen dimensions.
Here is a gif of the build on OSX. I set the window to be 1680x1050 and my computer's current resolution is 2880x1800. I had also test it on Windows but did not record it as the example looks nearly identical.
Let me know if you have more questions about the implementation or if there are issues with other platforms I did not test.
Edit: Just realized you want the screen space coordinate relative to the monitor space. I will correct the snippet in a little bit - in a meeting right now.
Edit2: After a bit more looking, it will not be easy to get the exact coordinates without the window being centered or getting the standalone window's position. I do not believe there is an easy way to get this information without a dll, so here is a implementation for mac and a solution for windows.
Currently, the solution I have will only get the screen position if the standalone player is windowed and centered on your screen. If the player is centered on the screen, I know that the center of my monitor is half the dimensions of its resolution, and know that the center point of my window matches up to this point. I can now get the bottom left corner of my window relative to my monitor and not a (0,0) coordinate. As the screen space has the bottom left corner at (0,0), you can now adjust the position to monitor space by adding the position of the newly calculated bottom left position.
Here is the new new GenerateData method:
/// <summary>
/// Onclick of our button to test generating data when the object moves
/// </summary>
public void GenerateData()
{
// the anchored position is relative to screen space if the canvas is an overlay - if not, it will need to be converted to screen space based on our camera
Vector3 screenPos = parentCanvas.renderMode == RenderMode.ScreenSpaceCamera ? mainCam.WorldToScreenPoint(transform.position) : rect.transform.position;
// grab the display dimensions
Vector2 displayDimensions;
// bug or something with mac or retina display on mac where the main.system dimensions are half of what they actually are
#if UNITY_STANDALONE_OSX || UNITY_EDITOR_OSX
displayDimensions = new Vector2(Display.main.systemWidth * 2f, Display.main.systemHeight * 2f);
#else
displayDimensions = new Vector2(Display.main.systemWidth, Display.main.systemHeight);
#endif
// the centerpoint of our display coordinates
Vector2 displayCenter = new Vector2(displayDimensions.x / 2f, displayDimensions.y / 2f);
// half our screen dimensions to find our screen space relative to monitor space
Vector2 screenDimensionsHalf = new Vector2(Screen.width / 2f, Screen.height / 2f);
// find the corners of our window relative to the monitor space
Vector2[] displayCorners = new Vector2[] {
new Vector2(displayCenter.x - screenDimensionsHalf.x, displayCenter.y - screenDimensionsHalf.y), // bottom left
new Vector2(displayCenter.x - screenDimensionsHalf.x, displayCenter.y + screenDimensionsHalf.y), // top left
new Vector2(displayCenter.x + screenDimensionsHalf.x, displayCenter.y + screenDimensionsHalf.y), // top right
new Vector2(displayCenter.x + screenDimensionsHalf.x, displayCenter.y - screenDimensionsHalf.y) // bottom right
};
for (int z = 0; z < 4; ++z)
{
text[z].text = displayCorners[z].ToString();
}
// outputting our screen position relative to our monitor
text[4].text = (new Vector2(screenPos.x, screenPos.y) + displayCorners[0]).ToString();
}
Once you are able to either get or set the windowed screen, you can properly re-orient the lower-left corner relative to the monitor dimensions or you can set the window back to the center point of your monitor. The above snippet would also work for a full-screen player. You would just need to determine how far off the aspect ratio of the player window is to your monitor, which allows you to find how large the black bars would be on the edges.
I assumed what you had wanted was straightforward but from what I can tell an OS-agnostic solution would be difficult. My above solution should work for any platform when the player is windowed if you can either get or set the standalone window position and for any platform that is full-screened with the theoretical approach I mentioned.
If you want more info on how to adjust the implementation for the full-screened window let me know.

Unity GUI Label show twice for no reason

I'm working on an application where several GUI Label display names of planes.
And here's the result :
The problem is, if I rotate my camera by 180 °, those label are here, like there is a point symmetry :
So my label appear twice, once on the plane, which is good, and a second time, behind the camera.
I check if my script was not added twice, but there is no problem, more strange, if I look from an above view, the problem just disappear :
I have no idea where this can come from, here's my code, who is attached to every plane :
void OnGUI()
{
if (showInfos)
{
Rect r = new Rect((Camera.main.WorldToScreenPoint(gameObject.transform.position)).x+25, Camera.main.pixelHeight - (Camera.main.WorldToScreenPoint(gameObject.transform.position)).y+25, 75f, 75f);
GUI.Label(r, gameObject.transform.root.name);
}
}

You are drawing the labels whether or not they are in the view frustum or not.
From Camera.WorldToScreenPoint (emphasis mine):
Screenspace is defined in pixels. The bottom-left of the screen is (0,0); the right-top is (pixelWidth,pixelHeight). The z position is in world units from the camera.
You need to check if the Z value of the screen point is negative or positive (I don't know which one is in front of cam and which one is behind it, I don't use Unity), and according to that decide if it needs to be rendered or not.
void OnGUI()
{
if (showInfos)
{
var pt = Camera.main.WorldToScreenPoint(gameObject.transform.position);
if (pt.z > 0) //or < 0, no idea.
{
Rect r = new Rect(pt.x + 25, Camera.main.pixelHeight - pt.y + 25, 75f, 75f);
GUI.Label(r, gameObject.transform.root.name);
}
}
}

game works perfectly in standard screensize but not in fullscreen

So a friend and I are making a horizontal scroller game. There are mice hidden throughout the screen and because it's a horizontal scroller game you can also scroll to the left and right and see more mice.
We originally made this game in de default xna screensize (800 x 480). We're now at a point where we want our game to be fullscreen. Our point of the game is to click on the mice and get a higher score, etc...
Now before we made our game fullscreen we could click on the mice and our score would add up. After we've made it fullscreen we can't click on the mice anymore. Well, we can still click on them, but instead of adding points to our score nothing happens.
We're using bounding boxes on the mice, and whenever the X & Y position of our computermouse is withing one of those boundingboxes and you will click with your computermouse at the moment that your computermouse is inside a mouse boundingbox then you're supposed to get points adding up to your score. This is the code for the boundingbox on every mouse.
public Rectangle getBoundingBox()
{
return new Rectangle(
(int)_position.X,
(int)_position.Y,
_texture.Width,
_texture.Height
);
}
We gave all our mice a position by the next code:
_position = _scrollerBackground.GetPosition() + _offsetFromBackground;
The next piece of code is how we get the position of our mouse and what it should do when the position of the mouse is within one of our mice(_number0). When the mouseposition is within the boundingbox from the mouse and when the computermouse clicks at that moment then input = 0;
public void UpdateMouse()
{
mouse = Mouse.GetState();
mousePosition.X = mouse.X;
mousePosition.Y = mouse.Y;
}
public void OnMouseOver()
{
if (_number0.getBoundingBox().Contains(mousePosition))
{
if (mouse.LeftButton == ButtonState.Pressed && oldMouseState.LeftButton == ButtonState.Released)
{
input += "0";
}
}
This is how it worked before we made our game fullscreen, but now that doesn't work anymore. This is the code we used to make our game fullscreen:
graphics.IsFullScreen = true;
Does anyone have any similar experiences or maybe knows how to fix this?

How to work out what direction an object is moving

SO I currently have an object which moves in the Y axis (up and down). How would I be able to program it so the program knows whether the object is moving up or down?
I understand I'd need an if statement like the following:
if (object is moving up)
{
//set direction to 1
}
else
{
//object must be going down, set direction to 2
}
I just don't understand what syntax I'd need to use. This would be easy if I was holding down the key and it was moving up or down however that's not the case. The object is a bouncing ball and therefore when you set a power the ball jumps, and bounces so it is constantly changing.
Thanks for your help, let me know if this wasn't described well and you need more info.

You need to save the current coordinates so that you can check them after the update method is called. Then you can check the differences between the previous and current location:
Declare it in Game1() constructor:
Point previous = new Point();
previous.X = myObject.InitialX;
previous.Y = myObject.InitialY;
In Update:
int deltaX = object.X - previous.X;
int deltaY = object.Y - previous.Y;
if (deltaX < 0)
{
object is moving upwards
}
else
{
object is moving downwards
}
if (deltaY < 0)
{
object is moving to the left
}
else
{
object is moving to the right
}
//update the previous state
previous.X = myObject.X;
previous.Y = myObject.Y;

When you update the position of the ball you have one position and you calculate the position for the next frame -> You can calculate the distance vector.
With the sign of the y-value of this distance vector you can decide if it's moving upwards, downwards or isn't moving (=0).

UI and unity position

I have this problem, that I want to be able to click on a "tile" on the screen and then a pop-up menu should be shown just next to the tile. I can click on the tile and then a pop-up menu shows up but not where I want it.
On the picture here I've clicked on the top left one.
My code for placing the picture is as following:
using UnityEngine;
using System.Collections;
public class TowerMenu : MonoBehaviour
{
bool showMenu = false;
float x;
float y;
GUIStyle myStyle;
public Texture2D[] towers;
void OnGUI()
{
if(showMenu)
{
//Bear tower
GUI.Button(new Rect(x + 10, y - 25, 50, 50), towers[0]);
//Seal tower
GUI.Button(new Rect(x + 10, y + 25, 50, 50), towers[1]);
}
}
public void ShowMenu(Vector2 pos)
{
showMenu = true;
x = pos.x;
y = pos.y;
}
}
Hope anyone can help me :)

Sry, i cant comment because i dont have enough rep, this is a comment to Steven Mills answer and comments to that post.
The first error comes because you are calling WorldToViewportPoint as if it was a static member function which it isnt(Think of if you had 2 Cameras you would have to specify which Camera you want, reading up on what a static member is would be helpful here). What you need to do to fix this is get a reference of your MainCamera and call the function from that instance(best method would probably be with a public variable and dragging the camera on the script in the editor)
The second error occurs because you are trying to give a Vector3 to ShowMenu() which requires a Vector2. The third is probably a product of the compiler trying to fix error 2
This is a logical error because the tiles are GameObjects and thus the transform.position are positions in 3d space. What you actually want is the 2d-pixel-position on the screen of your MainCamera. To get this you need to call WorldToScreenPoint instead of WorldToViewportPoint. Sadly, as you will notice you will also get a Vector3 here which is not what you want, but you can just take the x and y coordinates as your pixel screen coordinates. the z-coordinate denotes the distance from the camera.
Hope that clears it up a little instead of confusing you ;D
Feel free to ask again, but also try to read the Unity Script Reference and try to understand what is written there and what it means :)

By the look of it, your ShowMenu method is receiving a pos of (0,0), which is why the two buttons are placed at what seems to be a position of (10,-25) and (10,25) respectively.
Without seeing the code calling ShowMenu I can't be sure what location you're giving, but my guess would be that the tiles belong to a parent object, and you're passing the local position instead of the world position.
If you post the code in which you call ShowMenu I may be able to point out any problems.
EDIT:
Based on the information provided in the comments, the problem is that the position needs converting between the world coordinates and screen coordinates:
void OnMouseDown()
{
if(state == State.water)
{
errorHandler.sendError("You can't click on that");
}
if(state == State.ice)
{
towerMenu.ShowMenu(camera.WorldToViewportPoint(this.transform.position));
}
}
and change the ShowMenu to this:
public void ShowMenu(Vector3 pos)
{
showMenu = true;
x = pos.x;
y = pos.y;
}