Unity canvas scaler blocking GL.Vertex - c#

So I'm drawing lines using GL in Unity. I draw those lines on my screen, and they are being drawn. However, I only see the lines once I disable the canvas scaler on the script that is providing the data for the lines.
I tried other ways of drawing the lines but nothing helped.
So the setup is as followed, I have a canvas in my scene with a canvas scaler ontop of it that scales with screen size. This canvas contains a script that provides the data (positions) for the lines. When the lines need to be drawn, it fires an event with a vector2[] with all the data that needs lines etc. I have another script on my main camera listening to this event, and when needed it draws the lines in the onpostrender. These lines are being drawn at the correct positions, but I can only see them once I disable the canvas scaler on the canvas. Here's some code:
This is the script in my scene (thats on the canvas) that fires an event with the data.
lock (_skeleton2D)
{
for (int i = 0; i < skeleton.Length; ++i)
{
_skeleton2D[i] = new Vector2(skeleton[i].x * scaleX + _depthImageRect.anchoredPosition.x * 2, _depthImageRect.rect.height - skeleton[i].y * scaleY);
}
for (int i = 0; i < skeleton.Length; i++)
{
Vector2 vector2 = skeleton[i];
objects[i].transform.position = vector2;
// _skeleton2D[i] = vector2;
}
DrawSkeletonRenderer?.Invoke(_skeleton2D);
As you can see in the end it fires the DrawSkeletonRenderer with a Vector2[] _skeleton2D. Then I have a script on the Main and only camera in the scene which is listening to this event, and when it receives the event, it updates it's own Vector2[] of skeletondata, and uses this to render the lines.
private void Start()
{
_skeletonMaterial = new Material(Shader.Find("UI/Default"));
MoveCalibrationToolManager.DrawSkeletonRenderer += receivedDrawSkeleton;
}
private void receivedDrawSkeleton(Vector2[] skeleton)
{
_skeleton2D = skeleton;
isSkeletonValid = true; //TODO Turn false when not needed anymore.
}
private bool isSkeletonValid;
private void OnPostRender()
{
if (isSkeletonValid)
{
lock (_skeleton2D)
{
drawSkeleton(_skeleton2D);
Debug.LogError("DRAW SKELETON RIGHT NOW!");
}
}
}
As you can see it eventually calls drawSkeleton which as shown below draws the lines
GL.PushMatrix();
_skeletonMaterial.SetPass(0);
GL.LoadPixelMatrix();
drawBone(skeleton[(int)JointType.Head], skeleton[(int)JointType.Neck]);
drawBone(skeleton[(int)JointType.Neck], skeleton[(int)JointType.LShoulder]);
drawBone(skeleton[(int)JointType.Neck], skeleton[(int)JointType.RShoulder]);
drawBone(skeleton[(int)JointType.LShoulder], skeleton[(int)JointType.LElbow]);
drawBone(skeleton[(int)JointType.LElbow], skeleton[(int)JointType.LHand]);
drawBone(skeleton[(int)JointType.RShoulder], skeleton[(int)JointType.RElbow]);
drawBone(skeleton[(int)JointType.RElbow], skeleton[(int)JointType.RHand]);
drawBone(skeleton[(int)JointType.LShoulder], skeleton[(int)JointType.RWaist]);
Which then calls drawBone which looks as follows:
if (float.IsNaN(fromJoint.x) || float.IsNaN(fromJoint.y) || float.IsNaN(toJoint.x) || float.IsNaN(toJoint.y))
{
Debug.Log("Joint positions for lines are null");
return;
}
GL.Begin(GL.LINES);
GL.Color(Color.red);
GL.Vertex(fromJoint);
GL.Vertex(toJoint);
GL.End();
As I said, everything is drawn but I can only see it when I disable the Canvas Scaler on the canvas that has the first script attached.
During runtime, I make a texture on which I show a feed of a 3D realsense camera. I need the skeleton to be drawn ontop of that. I make this texture the following way:
GameObject obj = new GameObject("Calibration View");
obj.transform.SetParent(gameObject.transform, false);
obj.transform.SetAsFirstSibling();
_depthImage = obj.AddComponent<RawImage>();
_depthImageRect = obj.GetComponent<RectTransform>();
_depthImageRect.anchorMin = Vector2.zero;
_depthImageRect.anchorMax = Vector2.one;
_depthImageRect.anchoredPosition = Vector2.zero;
Quaternion transformRotation = new Quaternion(180f, 0f, 0f, 0f);
_depthImageRect.transform.rotation = transformRotation;
_depthImage.material = _depthImageMaterial;
_depthTexture = new Texture2D(RsDevice.DepthWidth, RsDevice.DepthHeight, TextureFormat.BGRA32, false);
_depthImage.texture = _depthTexture;
_depthTextureBytes = new byte[RsDevice.DepthWidth * RsDevice.DepthHeight * 4];

Related

Unity's SetPixel method does not color out the given pixels

I am currently developing a pixel art program in Unity. Obviously, it has a pencil tool with a script on it that I have made.
Unfortunately, the SetPixel method does not color the pixels. I don't know if it is the method itself that it's not working or something else.
This is the code I am using:
[SerializeField] private Sprite textureRendererSprite;
private Texture2D texture;
private MouseCoordinates mouseCoordinates;
void Start()
{
mouseCoordinates = GetComponent<MouseCoordinates>();
texture = textureRendererSprite.texture;
}
void Update()
{
if (Input.GetMouseButtonDown(0))
{
texture.SetPixel(int.Parse(mouseCoordinates.posInt.x.ToString()), int.Parse(mouseCoordinates.posInt.y.ToString()), Color.black);
Debug.Log(int.Parse(mouseCoordinates.posInt.x.ToString()));
Debug.Log(int.Parse(mouseCoordinates.posInt.y.ToString()));
}
}
Also, this is my MouseCoordinates script:
[SerializeField] private Canvas parentCanvas = null;
[SerializeField] private RectTransform rect = null;
[SerializeField] private Text text;
public Vector2 posInt;
[SerializeField] private Camera UICamera = null;
void Start()
{
if (rect == null)
rect = GetComponent<RectTransform>();
if (parentCanvas == null)
parentCanvas = GetComponentInParent<Canvas>();
if (UICamera == null && parentCanvas.renderMode == RenderMode.WorldSpace)
UICamera = parentCanvas.worldCamera;
}
public void OnPointerClick(PointerEventData eventData)
{
RectTransformUtility.ScreenPointToLocalPointInRectangle(rect, eventData.position, UICamera, out Vector2 localPos);
localPos.x += rect.rect.width / 2f;
localPos.y += rect.rect.height / 2f;
posInt.x = ((int)localPos.x);
posInt.y = ((int)localPos.y);
text.text = (posInt.x + ", " + posInt.y).ToString();
}
I was a little bored, so here is a fully working pixel draw I just whipped up. The one part you were missing with your implementation is Texture2D.Apply, which based on the Texture2D.SetPixels doc page,
This function takes a color array and changes the pixel colors of the
whole mip level of the texture. Call Apply to actually upload the
changed pixels to the graphics card.
Now to your actual implementation. You do not need a majority of the data you are caching, as a PointerEventData already has most of it. The only component you will need is the Image component that you want to change.
OnPointerClick is fine, but that only registers clicks, not dragging. If you want to make a pixel art tool, most art is done by dragging a cursor or stylus, so you will want to use an OnDragHandler instead or, along with your click.
One other note, you are not adding any brush size. More of a QoL update to your snippet, but with the addition of a brush size there are other complications that arise. SetPixel is bottom left aligned and must be contained within the bounds of the texture. You can correct this by offsetting the center point of your click by half a brush size, then clamping the width and height of your box.
Here is the current snippet:
using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.UI;
public class TestScript : MonoBehaviour, IPointerClickHandler, IDragHandler
{
// color we are setting pixels to
[SerializeField] private Color clr = Color.white;
// our source UI image - it can be a raw image or sprite renderer, I just used UI image
[SerializeField] private Image img = null;
[Range(1, 255)]
[SerializeField] private int BrushSize = 1;
// the texture we are going to manipulate
private Texture2D tex2D = null;
private void Awake()
{
Sprite imgSprite = img.sprite;
// create a new instance of our texture to not write to it directly and overwrite it
tex2D = new Texture2D((int)imgSprite.rect.width, (int)imgSprite.rect.height);
var pixels = imgSprite.texture.GetPixels((int)imgSprite.textureRect.x,
(int)imgSprite.textureRect.y,
(int)imgSprite.textureRect.width,
(int)imgSprite.textureRect.height);
tex2D.SetPixels(pixels);
tex2D.Apply();
// assign this new texture to our image by creating a new sprite
img.sprite = Sprite.Create(tex2D, img.sprite.rect, img.sprite.pivot);
}
public void OnPointerClick(PointerEventData eventData)
{
Draw(eventData);
}
public void OnDrag(PointerEventData eventData)
{
Draw(eventData);
}
private void Draw(in PointerEventData eventData)
{
Vector2 localCursor;
// convert the position click to a local position on our rect
if (!RectTransformUtility.ScreenPointToLocalPointInRectangle(img.rectTransform, eventData.position, eventData.pressEventCamera, out localCursor))
return;
// convert this position to pixel coordinates on our texture
int px = Mathf.Clamp(0, (int)((localCursor.x - img.rectTransform.rect.x) * tex2D.width / img.rectTransform.rect.width), tex2D.width);
int py = Mathf.Clamp(0, (int)((localCursor.y - img.rectTransform.rect.y) * tex2D.height / img.rectTransform.rect.height), tex2D.height);
// confirm we are in the bounds of our texture
if (px >= tex2D.width || py >= tex2D.height)
return;
// debugging - you can remove this
// print(px + ", " + py);
// if our brush size is greater than 1, then we need to grab neighbors
if (BrushSize > 1)
{
// bottom - left aligned, so find new bottom left coordinate then use that as our starting point
px = Mathf.Clamp(px - (BrushSize / 2), 0, tex2D.width);
py = Mathf.Clamp(py - (BrushSize / 2), 0, tex2D.height);
// add 1 to our brush size so the pixels found are a neighbour search outward from our center point
int maxWidth = Mathf.Clamp(BrushSize + 1, 0, tex2D.width - px);
int maxHeight = Mathf.Clamp(BrushSize + 1, 0, tex2D.height - py);
// cache our maximum dimension size
int blockDimension = maxWidth * maxHeight;
// create an array for our colors
Color[] colorArray = new Color[blockDimension];
// fill this with our color
for (int x = 0; x < blockDimension; ++x)
colorArray[x] = clr;
// set our pixel colors
tex2D.SetPixels(px, py, maxWidth, maxHeight, colorArray);
}
else
{
// set our color at our position - note this will almost never be seen as most textures are rather large, so a single pixel is not going to
// appear most of the time
tex2D.SetPixel(px, py, clr);
}
// apply the changes - this is what you were missing
tex2D.Apply();
// set our sprite to the new texture data
img.sprite = Sprite.Create(tex2D, img.sprite.rect, img.sprite.pivot);
}
}
Here is a gif of the snippet in action. Quite fun to play around with. And remember, whatever texture you use for this must have the setting Read and Write enabled on the import settings. Without this setting, the data is not mutable and you can not access the texture data at runtime.
Edit: Skimmed your question a bit too quickly. Realizing you are using a 2D sprite and not a UI Image or RawImage. You can still draw to a Sprite, but as it is not a UI object, it does not have a RectTransform. However, in your second snippet you reference a RectTransform. Can you explain your setup a bit more? The answer I provided should be enough to point you in the right direction either way.

Create a new photo at the tapped point

I want to create a new photo at the point I touch it and I want it to be done with every touch so I wrote the following line inside the void Update () function.
public Canvas cv;
public Image im;
I have defined the UI elements above.
for (var i = 0; i < Input.touchCount; ++i)
{ Touch touch = Input.GetTouch(i);
if (touch.phase == TouchPhase.Began)
{
Instantiate(im, Input.GetTouch(i).position, Quaternion.identity).transform.SetParent(cv.transform, false);
}
}
And when I try it with the unity remote app, I take the picture about 3-4 fingers above the point I click. what's the problem? please help!
Vector2 scaleSomething()
{
var screenPosition = Camera.main.WorldToScreenPoint(worldPosition);
var scaler = cv.GetComponentInParent<CanvasScaler>();
var guiScale = 1.0f;
if (Mathf.Approximately(scaler.matchWidthOrHeight, 0.0f))
guiScale = scaler.referenceResolution.x / (float) Screen.width;
else if (Mathf.Approximately(scaler.matchWidthOrHeight, 1.0f))
guiScale = scaler.referenceResolution.y / (float) Screen.height;
return new Vector2(
(screenPosition.x - (Screen.width* 0.5f)) * guiScale,
(screenPosition.y - (Screen.height* 0.5f)) * guiScale);
}
try removing the set parent part after instantiating
edit what is happening is when you instantiate your objects as children they get translated/scaled in local space to the parent. Because your canvas is getting stretched (scaled up on y) your children elements are also getting scaled up on y and are out of place.

C# Xna achieve a desent scale effect with sprite fonts

So I am working on a menu for my game. Here's the code for a single button:
// button class
public class ButtonGUI
{
public SpriteFont spriteFont;
string btnTxt;
public Rectangle btnRect;
Color colour;
public ButtonGUI(string newTxt, Rectangle newRect, SpriteFont newSpriteFont, Color newColour)
{
spriteFont = newSpriteFont;
btnRect = newRect;
btnTxt = newTxt;
colour = newColour;
}
public void Draw(SpriteBatch spriteBatch)
{
// TextOutliner() is a static helper class for drawing bordered spritefonts I made
TextOutliner.DrawBorderedText(spriteBatch, spriteFont, btnTxt, btnRect.X, btnRect.Y, colour);
}
}
// TitleScreen.cs
ButtonGUI btnPlay, btnPlay_2;
bool indexPlay;
string[] menuTxt;
SpriteFont GameFontLarge, GameFontLargeHover;
// LoadContent() method:
// Load both spritefonts....
menuTxt = new string[4];
menuTxt[0] = "Play Game";
menuTxt[1] = "Achievements";
menuTxt[2] = "Settings";
menuTxt[3] = "Exit Game";
btnPlay = new ButtonGUI(menuTxt[0], new Rectangle(150, 300, (int)GameFontLarge.MeasureString(menuTxt[0]).X, (int)GameFontLarge.MeasureString(menuTxt[0]).Y), GameFontLarge, Color.White);
btnPlay_2 = new ButtonGUI(menuTxt[0], new Rectangle(150, 300, (int)GameFontLargeHover.MeasureString(menuTxt[0]).X, (int)GameFontLargeHover.MeasureString(menuTxt[0]).Y), GameFontLargeHover, Color.Yellow);
// Update() method:
MouseState mouseState = Mouse.GetState();
Rectangle mouseRect = new Rectangle(mouseState.X, mouseState.Y, 1, 1);
if (mouseRect.Intersects(btnPlay.btnRect))
{
indexPlay = true;
if (mouseState.LeftButton == ButtonState.Pressed) Game1.CurrentGameState = Game1.GameState.playScreen;
}
else indexPlay = false;
// Draw() method:
if (indexPlay)
{
btnPlay_2.Draw(spriteBatch);
}
else btnPlay.Draw(spriteBatch);
So I do all of this for 4 different buttons. The 2 sprite fonts are of the same font but with different size. Now when I test the game, when I mouse-hover on each of the buttons, the text changes from white to yellow and becomes larger. But since the button coordinates are done with x = left-most; y = top-most when the font changes the bigger font is drawn at the same position as the smaller one was. I want to get a decent "scale" effect when mouse-hovering. I mean I know I set the button position to that when initializing, but still I need some type of algorithm to find a way and draw the bigger font on the center of the older... What would be your approach?
You could change the position of your hover button based on the size difference:
var sizeDifference = btnPlay_2.btnRect.Size - btnPlay.btnRect.Size;
btnPlay_2.btnRect.Offset(-sizeDifference.X/2, -sizeDifference.Y/2);

How to shoot a triangle out of an asteroid which floats all of the way up to the screen?

I currently have an asteroid texture loaded as my "test player" for the game I'm writing. What I'm trying to figure out how to do is get a triangle to shoot from the center of the asteroid, and keep going until it hits the top of the screen. What happens in my case (as you'll see from the code I've posted), is that the triangle will show, however it will either be a long line, or it will just be a single triangle which stays in the same location as the asteroid moving around (that disappears when I stop pressing the space bar), or it simply won't appear at all. I've tried many different methods, but I could use a formula here.
All I'm trying to do is write a space invaders clone for my final in C#. I know how to code fairly well, my formulas just need work is all.
So far, this is what I have:
Main Logic Code
protected override void Draw(GameTime gameTime)
{
GraphicsDevice.Clear(ClearOptions.Target, Color.Black, 1, 1);
mAsteroid.Draw(mSpriteBatch);
if (mIsFired)
{
mPositions.Add(mAsteroid.LastPosition);
mRay.Fire(mPositions);
mIsFired = false;
mRay.Bullets.Clear();
mPositions.Clear();
}
base.Draw(gameTime);
}
Draw Code
public void Draw()
{
VertexPositionColor[] vertices = new VertexPositionColor[3];
int stopDrawing = mGraphicsDevice.Viewport.Width / mGraphicsDevice.Viewport.Height;
for (int i = 0; i < mRayPos.Length(); ++i)
{
vertices[0].Position = new Vector3(mRayPos.X, mRayPos.Y + 5f, 10);
vertices[0].Color = Color.Blue;
vertices[1].Position = new Vector3(mRayPos.X - 5f, mRayPos.Y - 5f, 10);
vertices[1].Color = Color.White;
vertices[2].Position = new Vector3(mRayPos.X + 5f, mRayPos.Y - 5f, 10);
vertices[2].Color = Color.Red;
mShader.CurrentTechnique.Passes[0].Apply();
mGraphicsDevice.DrawUserPrimitives<VertexPositionColor>(PrimitiveType.TriangleStrip, vertices, 0, 1);
mRayPos += new Vector2(0, 1f);
mGraphicsDevice.ReferenceStencil = 1;
}
}
This isn't quite how you're supposed to be manipulating the location of a model in world space and since you're creating a new vertex array every single draw frame you'll find that it performs pretty badly when you come to draw more than a few triangles.
declare the vertices and index list for your triangle just once in the LoadContent method.
VertexBuffer triangleVertexBuffer;
IndexBuffer triangleIndexBuffer;
protected override void LoadContent()
{
// Setup a basic effect to draw the triangles with.
mEffect = new BasicEffect(GraphicsDevice);
// setup the triangle vertext buffer and load up it's content.
triangleVertexBuffer = new VertexBuffer(GraphicsDevice, typeof(VertexPositionColor), 3, BufferUsage.WriteOnly);
triangleVertexBuffer.SetData<VertexPositionColor>(new VertexPositionColor[]
{
new VertexPositionColor (new Vector3 (0f, -1f, 0.0f), Color.Blue), // Top Point
new VertexPositionColor (new Vector3 (-1f, 1f, 0.0f), Color.White), // Bottom Left
new VertexPositionColor (new Vector3 (1f, 1f, 0.0f), Color.Red), // Bottom Right
});
// setup an index buffer to join the dots!
triangleIndexBuffer = new IndexBuffer(GraphicsDevice, IndexElementSize.SixteenBits, 3, BufferUsage.WriteOnly);
triangleIndexBuffer.SetData<short>(new short[]
{
0,
1,
2,
});
}
After this assuming your effect takes in to account a world transformation (basic effect does) you can use that parameter to move the triangle.
protected override void Draw(GameTime gameTime)
{
GraphicsDevice.Clear(Color.CornflowerBlue);
for (int i = 0; i < mRayPos.Length ; i++)
{
// This is the line that moves the triangle along your ray.
mEffect.World = Matrix.CreateTranslation(new Vector3(mRayPos[i].X, mRayPos[i].Y, mRayPos[i].Z));
mEffect.Techniques[0].Passes[0].Apply();
// These lines tell the graphics card that you want to draw your triangle.
GraphicsDevice.SetVertexBuffer(triangleVertexBuffer);
GraphicsDevice.Indices = triangleIndexBuffer;
GraphicsDevice.DrawIndexedPrimitives(PrimitiveType.TriangleList, 0, 0, 3, 0, 1);
}
base.Draw(gameTime);
}
If you use this method it becomes a very simple operation to rotate or scale your trangle using Matrix.CreateRotation and Matrix.CreateScale.

How can I get an IBeam cursor in an XNA Game?

I've managed to get the cursor to change to an IBeam in a trivial XNA 'Game' with Update as:
protected override void Update(GameTime gameTime)
{
// Allows the game to exit
if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
this.Exit();
// TODO: Add your update logic here
if (Keyboard.GetState().IsKeyDown(Keys.A))
{
System.Windows.Forms.Cursor.Current = System.Windows.Forms.Cursors.Arrow;
}
if (Keyboard.GetState().IsKeyDown(Keys.I))
{
System.Windows.Forms.Cursor.Current = System.Windows.Forms.Cursors.IBeam;
}
base.Update(gameTime);
}
The mouse changes to the IBeam cursor when 'I' is pressed, but immediately changes back to an arrow when you move the mouse. Is there a way to get it to stay as the default Windows IBeam, or will I need to create and track a custom cursor?
[EDIT] I should also point out that setting the cursor every single frame causes it to flicker when the mouse is moved. It seems the XNA (or Windows) is internally resetting the cursor to an arrow every frame?
Before resorting to drawing it manually, I'd try to set the Cursor property of the underlying System.Windows.Forms.Form:
Form f = (Form)Control.FromHandle(Game.Window.Handle);
f.Cursor = System.Windows.Forms.Cursors.IBeam;
As I don't have XNA installed right now, I don't know if this will work, or if it will be permenant, but I don't see why it wouldn't.
Sounds like your going to have to draw it manually. It shouldn't be too hard - just draw a sprite via SpriteBatch at the location of the cursor.
// ex.
protected override void Draw(GameTime gameTime)
{
// Other stuff...
base.Draw(gameTime);
// Retrieve mouse position
MouseState mState = Mouse.GetState();
Vector2 mousePos = new Vector2(mState.X, mState.Y);
// Use this instead to optionally center the texture:
// Vector2 mousePos = new Vector2(mState.X - cursorTexture.Width / 2, mState.Y - cursorTexture.Height / 2);
// Draw cursor after base.Draw in order to draw it after any DrawableGameComponents.
this.spriteBatch.Begin(); // Optionally save state
this.spriteBatch.Draw(cursorTexture, mousePos, Color.White);
this.spriteBatch.End();
}
And here's some code to extract the cursor image. Keep in mind you'll need an extra reference to System.Drawing.
protected override void LoadContent()
{
// Other stuff...
// The size of the cursor in pixels.
int cursorSize = 32;
// Draw the cursor to a Bitmap
Cursor cursor = Cursors.IBeam;
Bitmap image = new Bitmap(cursorSize, cursorSize);
Graphics graphics = Graphics.FromImage(image);
cursor.Draw(graphics, new Rectangle(0, 0, cursorSize, cursorSize));
// Extract pixels from the bitmap, and copy into the texture.
cursorTexture = new Texture2D(GraphicsDevice, cursorSize, cursorSize);
Microsoft.Xna.Framework.Graphics.Color[] data = new Microsoft.Xna.Framework.Graphics.Color[cursorSize * cursorSize];
for (int y = 0; y < cursorSize; y++)
{
for (int x = 0; x < cursorSize; x++)
{
System.Drawing.Color color = image.GetPixel(x, y);
data[x + y * cursorSize] = new Microsoft.Xna.Framework.Graphics.Color(color.R, color.G, color.B, color.A);
}
}
cursorTexture.SetData<Microsoft.Xna.Framework.Graphics.Color>(data);
}
Keep in mind this is off the top of my head - not guaranteed to work. The basic idea is there, though.

Categories