I've follow the guide I find on https://learnopengl.com/ and managed to render a triangle on the window, but this whole process just seems over complicated to me. Been searching the internet, found many similar questions but they all seem outdated and none of them gives a satisfied answer.
I've read the guide Hello Triangle from the site mentioned above which has a picture explaining the graphics pipeline, with that I got the idea about why a seemingly simple task as drawing a triangle shape on screen takes so many steps.
I have also read the guide Coordinate Systems from the same site, which tells me what is the "strange (to me) coordinate" that OpenGL uses (NDC) and why it uses that.
(Here's the picture from the guide mentioned above which I think would be useful for describing my question.)
The question is: Can I use the final SCREEN SPACE coordinates directly?
All I want is do some 2D rendering (no z-axis) and the screen size is known (fixed), as such I don't see any reason why should I use a normalized coordinate system instead of a special one bound to my screen.
eg: on a 320x240 screen, (0,0) represents the top-left pixel and (319,239) represents the bottom-right pixel. It doesn't need to be exactly what I describe it, the idea is every integer coordinate = a corresponding pixel on the screen.
I know it's possible to setup such a coordinate system for my own use, but the coordinates would be transformed all around and in the end back to screen space - which is what I have in the first place. All these just seems to be wasted work to me, also is't it gonna introduce precision lost when the coordinates get transformed?
Quote from the guide Coordinate Systems (picture above):
After the coordinates are in view space we want to project them to clip coordinates. Clip coordinates are processed to the -1.0 and 1.0 range and determine which vertices will end up on the screen.
So consider on a 1024x768 screen, I define the Clip coordinates as (0,0) to (1024,678), where:
(0,0)--------(1,0)--------(2,0)
| | |
| First | |
| Pixel | |
| | |
(0,1)--------(1,1)--------(2,1) . . .
| | |
| | |
| | |
| | |
(0,2)--------(1,2)--------(2,2)
.
.
.
(1022,766)---(1023,766)---(1024,766)
| | |
| | |
| | |
| | |
(1022,767)---(1023,767)---(1024,767)
| | |
| | Last |
| | Pixel |
| | |
(1022,768)---(1023,768)---(1024,768)
Let's say I want to put a picture at Pixel(11,11), so the clip coordinates for that would be Clip(11.5,11.5) this coordinate is then processed to the -1.0 and 1.0 range:
11.5f * 2 / 1024 - 1.0f = -0.977539063f // x
11.5f * 2 / 768 - 1.0f = -0.970052063f // y
And I have NDC(-0.977539063f,-0.970052063f)
And lastly we transform the clip coordinates to screen coordinates in a process we call viewport transform that transforms the coordinates from -1.0 and 1.0 to the coordinate range defined by glViewport. The resulting coordinates are then sent to the rasterizer to turn them into fragments.
So take the NDC coordinate and transform it back to screen coordinate:
(-0.977539063f + 1.0f) * 1024 / 2 = 11.5f // exact
(-0.970052063f + 1.0f) * 768 / 2 = 11.5000076f // error
The x-axis is accurate as 1024 is power of 2, but since 768 isn't so y-axis is off. The error is very little, but it's not exactly 11.5f, so I guess there would be some sort of blending happen instead of 1:1 representation of the original picture?
To avoid the rounding error mentioned above, I did something like this:
First I set the Viewport size to a size larger than my window, and make both width and height a power of two:
GL.Viewport(0, 240 - 256, 512, 256); // Window Size is 320x240
Then I setup the coordinates of vertices like this:
float[] vertices = {
// x y
0.5f, 0.5f, 0.0f, // top-left
319.5f, 0.5f, 0.0f, // top-right
319.5f, 239.5f, 0.0f, // bottom-right
0.5f, 239.5f, 0.0f, // bottom-left
};
And I convert them manually in the vertex shader:
#version 330 core
layout (location = 0) in vec3 aPos;
void main()
{
gl_Position = vec4(aPos.x * 2 / 512 - 1.0, 0.0 - (aPos.y * 2 / 256 - 1.0), 0.0, 1.0);
}
Finally I draw a quad, the result is:
This seem to produce a correct result (the quad has a 320x240 size), but I wonder if it's necessary to do all these.
What's the drawback of my approach?
Is there a better way to achieve what I did?
It seems rendering in wireframe mode hides the problem. I tried to apply texture to my quad (actually I switched to 2 triangles), and I got different result on different GPU and non of them seem correct to me:
Left: Intel HD4000 | Right: Nvidia GT635M (optimus)
I set GL.ClearColor to white and disabled the texture.
While both result fills the window client area (320x240), Intel give me a square sized 319x239 placed to the top left while Nvidia gives me a square sized 319x239 placed on the bottom left.
This is what it looks like with texture turned on.
The texture:
(I have it flipped vertically so I can load it easier in code)
The vertices:
float[] vertices_with_texture = {
// x y texture x texture y
0.5f, 0.5f, 0.5f / 512, 511.5f / 512, // top-left
319.5f, 0.5f, 319.5f / 512, 511.5f / 512, // top-right
319.5f, 239.5f, 319.5f / 512, 271.5f / 512, // bottom-right ( 511.5f - height 240 = 271.5f)
0.5f, 239.5f, 0.5f / 512, 271.5f / 512, // bottom-left
};
Now I'm completely stuck.
I thought I'm placing the quad's edges on exact pixel center (.5) and I'm sampling the texture also at exact pixel center (.5), yet two cards give me two different result and non of them is correct (zoom in and you can see the center of the image is slightly blurred, not a clear checker-board pattern)
What am I missing?
I think I figured out what to do now, I've posted the solutions as an answer and leave this question here for reference.
Okay.... I think I finally get everything work as I expect. The problem was that I forgot pixels have size - it's so obvious now that I can't understand why I missed that.
In this answer, I'll refer the Clip Space coordinates as Clip(x,y), where x/y ranges from 0 to screen width/height respectively. For example, on a 320x240 screen the Clip Space is from Clip(0,0) to Clip(320,240)
Mistake 1:
When trying to draw a 10 pixels square, I drew it from Clip(0.5,0.5) to Clip(9.5,9.5) .
It's true that those are the coordinates for the pixel centers of the begin (top-left) pixel and end (bottom-right) pixel of the square, but the real space the square takes is not from the pixel center of it's begin pixel and end pixel.
Instead, the real space said square takes is from the top-left corner of the begin pixel to the bottom-right corner of the end pixel. So, the correct coordinates I should use for the square is Clip(0,0) - Clip(10, 10)
Mistake 2:
Since I got the size of the square wrong, I was trying to map the texture wrong as well. Since now I have the square size fixed, I'll just fix the coordinates for the textures accordingly as well.
However, I found a better solution: Rectangle Texture , from which I quote:
When using rectangle samplers, all texture lookup functions automatically use non-normalized texture coordinates. This means that the values of the texture coordinates span (0..W, 0..H) across the texture, where the (W,H) refers to its dimensions in texels, rather than (0..1, 0..1).
This is VERY convenient for 2D rendering, for one I don't have to do any coordinates transforms, and for a bonus I don't need to flip the texture vertically anymore.
I tried it and it works just as I expected, but I come across a new problem: bleeding on edge when the texture is not placed on exact pixel grids.
Solving the bleeding:
If I use different textures for every square, I can avoid this problem by having the sampler clamp everything outside the texture with TextureWrapMode.ClampToEdge
However, I'm using a texture atlas, a.k.a "sprite sheet". I've searched the internet found solutions like:
Manually add padding to each sprite, basically leave safe spaces for bleeding error.
This is straight forward but I really don't like it as I'll loose the ability to tightly pack my texture, and it makes calculating the texture coordinates more complex also it just a waste of space anyway.
Use GL_NEAREST for GL_TEXTURE_MIN_FILTER and shift the coordinates by 0.5/0.375
This is very easy to code and it works fine for pixel arts - I don't want liner filter to make them blurry any way. But I'd also like to keep the ability to display a picture and move it around smoothly rather than jumping from pixel to pixel, so I need to have the ability to use GL_LINEAR.
One solution: Manually clamp the texture coordinates.
It's basically the same idea as TextureWrapMode.ClampToEdge, but at a per sprite basic rather than only on the edges of entire sprite sheet. I coded the fragment shader like this (just a proof-of-concept, I certainly need to improve it):
#version 330 core
out vec4 FragColor;
in vec2 TexCoord;
uniform sampler2DRect texture1;
void main()
{
vec2 mTexCoord;
mTexCoord.x = TexCoord.x <= 0.5 ? 0.5 : TexCoord.x >= 319.5 ? 319.5 : TexCoord.x;
mTexCoord.y = TexCoord.y <= 0.5 ? 0.5 : TexCoord.y >= 239.5 ? 239.5 : TexCoord.y;
FragColor = texture(texture1, mTexCoord);
}
(Yes, the "sprite" I use in this case is 320x240 which takes my entire screen.)
It's soooo easy to code thanks to the fact that Rectangle Texture uses non-normalized coordinates. It works well enough and I just called it a day and wrapped it up here.
Another solution (untested yet): Use Array Texture instead of texture atlas
The idea is simple, just set TextureWrapMode.ClampToEdge and have the sampler do it's job. I haven't look further into it but it seems to work in concept. Yet I really like the way coordinates work with Rectangle Texture and would like to keep it if possible.
Rounding Error
When trying to animate my square on the screen, I come to a very strange result: (notice the lower left corner of the square when the values on the left reads X.5)
This only happens on my iGPU (Intel HD4000) and not the dGPU (Nvidia GT635M via optimus). It looks like this because I fitted all coordinates to pixel center (.5) in fragment shader:
#version 330 core
out vec4 FragColor;
in vec2 TexCoord;
uniform sampler2DRect texture1;
void main()
{
vec2 mTexCoord;
// Clamp to spirte
mTexCoord.x = clamp(TexCoord.x, 0.5, 319.5);
mTexCoord.y = clamp(TexCoord.y, 0.5, 239.5);
// Snap to pixel
mTexCoord.xy = trunc(mTexCoord.xy) + 0.5;
FragColor = texture(texture1, mTexCoord);
}
My best guess is that the iGPU and dGPU rounds differently when converting coordinates to NDC (and back to screen coordinates)
Using a quad and a texture, both sized power-of-2 would avoid this problem. There's also workarounds like adding a small amount (0.0001 is enough on my laptop) to mTexCoord.xy before trunc.
Update: the solution
Okay, after a good sleep I come up with a relatively simple solution.
When dealing with pictures, nothing needs to be changed (let linear filter do it's job)
Since there's always gonna be rounding error, I basically just give up at this point and live with it. It won't be noticeable to human eyes any way.
When trying to fit the pixels in texture into pixel grid on screen, in addition to snapping the texture coordinates in fragment shader (as seen above), I also have to pre-shift the texture coordinates in vertex shader:
#version 330 core
layout (location = 0) in vec2 aPos;
layout (location = 1) in vec2 aTexCoord;
out vec2 TexCoord;
uniform mat4 projection;
void main()
{
TexCoord = aTexCoord + mod(aPos , 1);
gl_Position = projection * vec4(aPos.xy, 0.0, 1.0);
}
The idea is simple: when the square is placed at Clip(10.5, 10.5), shift the texture coordinate to Pixel(0.5, 0.5) as well. Of course this would mean the end coordinate gets shifted out of sprite region Pixel(320.5, 320.5) but this is fixed in fragment shader using clamp so no need to worry about it. Any coordinates in between (like Pixel(0.1, 0.1)) wound be snapped to Pixel(0.5, 0.5) by fragment shader as well, which creates a pixel-to-pixel result.
This gives me a consistent result across my iGPU(intel) and dGPU(Nvidia):
Positioned at Clip(10.5,10.5), notice the artifacts in lower-left corner is gone.
To sum it up:
Setup the Clip Space coordinates as a 1:1 represent to the screen.
Remember that pixels have size when calculating the size of a sprite.
Use the correct coordinates for textures and fix the bleeding on sprite edges.
When trying to snap texture pixels to screen pixels, take extra care about vertex coordinates in addition to texture coordinates.
Can I use the final SCREEN SPACE coordinates directly
No you can't. You have to transform the coordinates. Either on the CPU or in the shader (GPU).
If you want to use window coordinates, then you've to set an Orthographic projection matrix, which transforms the coordinates from x: [-1.0, 1.0], y: [-1.0, 1.0] (normalized device space) to your window coordinates x: [0, 320], y: [240, 0].
e.g. by the use of glm::ortho
glm::mat4 projection = glm::orhto(0, 320.0f, 240.0f, 0, -1.0f, 1.0f);
e.g. OpenTK Matrix4.CreateOrthographic
OpenTK.Matrix4 projection =
OpenTK.Matrix4.CreateOrthographic(0, 320.0f, 240.0f, 0, -1.0f, 1.0f);
In the vertex shader, the vertex coordinate has to be multiplied by the projection matrix
in vec3 vertex;
uniform mat4 projection;
void main()
{
gl_Position = projection * vec4(vertex.xyz, 1.0);
}
For the sake of completeness Legacy OpenGL, glOrtho:
(do not use the old and deprecated legacy OpenGL functionality)
glOrtho(0.0, 320.0, 240.0, 0.0, -1.0, 1.0);
As others have mentioned you need an orthographic projection matrix.
You can either implement it yourself by following guides such as these:
http://learnwebgl.brown37.net/08_projections/projections_ortho.html
Or maybe the framework you're using already has it.
If you set the right/left/top/bottom values to match your screen resolution then a point of coordinate x,y (z being irrelevant, you can use a 2d vector with a 4x4 matrix) will become the same x,y on the screen.
You can move the view around like a camera if you multiply this projection matrix by a translation matrix (first the translation then the projection).
Then you pass this matrix to your shader and multiply the position of the vertex by it.
As I have specified in the title, I have started to work on a simple application what contains in the main frame window, a double buffered panel. Inside this panel some graphics can be drawn, let's consider this panel is a simple viewport for the elements drawn inside.
Two functionalities were added here, pan and zoom what can scale transform and translate transform inside paint event using a delta updated on MouseDown and Move events and OnMouseWheel for updating the scale transform.
The real problem has arrived after trying to add a functionality to support creation of a Node (graphic element) inside viewport to a precise location when zoom is greater than 1 (scale 100%).
Scenario 1 -> The yellow rectangle was created correctly and is positioned exactly at the mouse pointer location, as in shown the image below (scale == 1).
Scenario 2 -> The yellow rectangle is highly shifted relative to mouse position with a viewport scale of aprox. 40%, as in the image below (scale == 1.4). The red filled circle was the mouse pressed (the cursor position remains unchanged, only a zoom was made).
Test scenario -> I have tried a lot of methods without success before posting there, this is one of them:
I really appreciate any kind of inputs, even ideas related to changing the OXY graph approach (as the translate functions use relative coordinates).
Maybe this will help:
PointF ScaleUnscale(PointF p, float offX, float offY, float sX, float sY, bool scale)
{
PointF pf = p;
if (scale)
{
pf = new PointF( ( p.X - offX )/ sX, (p.Y - offY)/ sY) ;
}
else
{
pf = new PointF( p.X * sX + offX, p.Y * sY + offY);
}
return pf;
}
This scales a mouse point to a canvas point or back:
cPoints.Add(ScaleUnscale(e.Location, .., true));
Tested with a translated and scaled Graphics object:
g.TranslateTransform(offX, offY);
g.ScaleTransform(scaleX, scaleY);
Trying to figure out a good solution for handling different resolutions and resizing in a 2D side-scrolling shooter game build with OpenTK (OpenGL).
Ortho is setup as follows.
private void Setup2DGraphics()
{
const double halfWidth = 1920 / 2;
const double halfHeight = 1080 / 2;
GL.MatrixMode(MatrixMode.Projection);
GL.LoadIdentity();
GL.Ortho(-halfWidth, halfWidth, -halfHeight, halfHeight, -1.0, 1.0);
GL.MatrixMode(MatrixMode.Modelview);
GL.LoadIdentity();
}
Ortho fixes to 1920*1080 to keep the ortho fixed size. Changing size would change game behaviour. Objects movement is set to pixels/sec etc.
public void ResizeWindow(int width, int height)
{
var screenSize = GetOptimalResolution(width, height);
var offsetHeight = screenSize.Y < height ? (height - screenSize.Y) / 2 : 0;
var offsetWidth = screenSize.X < width ? (width - screenSize.X) / 2 : 0;
// Set the viewport
GL.Viewport((int)offsetWidth, (int)offsetHeight, (int)screenSize.X, (int)screenSize.Y);
}
GetOptimalResolution returns the viewport size in order to maintain a 16:9 aspect ratio. Example: If screensize is 1024*768, it resturns 1024*576 (shrinking height). It then places the ortho higher in the viewport using half of the height difference as offset. This results in black bars above and below the windows.
This setup works. It prevents aspect ratio issues on all resolutions. However I'm questioning myself if this is the right way to do it? The issues I can think of with this design:
- On resolutions lower then 1920*1080, the view is scaled down. This has an effect of how sprites look. For example, text starts to look horrible when scaled down.
- If turning down resolution for performance, will this have an effect if the ortho stays the same size?
Did a lot of searching on this topic but so far the only things I can find is plain instructions on how to perform actions, rather then to build solutions. I also found out that using Mipmap's might solve my scaling issue. However the question remains, am I on the right track? Should I look at the aspect ratio problem from a different angle?
For example, text starts to look horrible when scaled down.
Don't use textures for text. Use a technique like signed distance fields for rendering clear text at any resolution.
For artwork, the ideal would be to have resolution specific artwork for a variety of common resolutions / DPIs. Failing that, mipmaps will drastically improve the quality of the rendered texture if it's scaled down from the source texture image.
It also might be easier to work the problem if you stop using pixel coordinates to do stuff. The default OpenGL screen coordinates are (-1,-1) lower left to (1,1) upper right. However, you can alter these using an ortho transform to account for aspect ratio like this
const double aspect = windowWidth / windowHeight;
if (aspect > 1.0) {
GL.Ortho(-1 * aspect, 1 * aspect, -1, 1, -1.0, 1.0);
} else {
GL.Ortho(-1, 1, -1 / aspect, 1 / aspect, -1.0, 1.0);
}
This produces a projection matrix that is at least 1 in both dimensions. This lets you treat the screen as a 2x2 square with the origin in the middle, and any additional stuff the viewer can see because of the aspect is cake.
I have a button class which is basic, it glows when mouse is over the button and works for my main menu. However with a pause function that I have the x values are going to be different all the time. I have a camera class which changes the viewport and will scroll depending on where the player's position is. This is all working fine. However when I introduced the pause option either the Drawn "button" is placed correctly, however the rectangle used for mouse detection is some distance away from that.
replay.SetPosition(new Vector2(camera.viewport.Width / 4 + 50,camera.centre.Y));
//replay.SetPosition(new Vector2((int)camera.centre.X , (int)camera.centre.Y);
replay.Update(mouse);
The commented out one draws the image correctly but the rectangle is way off.
The uncommented version has the correct rectangle placement but the image stays at currently 200(x),250(y)
if (isPaused)
{
spriteBatch.Draw(pauseTexture,pauseRectangle, Color.White);
//replay.Draw(spriteBatch);
exit.Draw(spriteBatch);
mainmenu.Draw(spriteBatch);
}
The camera class update elements:
public void Update(Vector2 position, int xOffset, int yOffset)
{
if (position.X < viewport.Width / 4)
centre.X = viewport.Width / 4;
else if (position.X > xOffset - (viewport.Width / 4))
centre.X = xOffset - (viewport.Width / 4);
else centre.X = position.X;
}
I will be around most of the day to add information if needed. Thanks for all the help!
Unfortunately I don't have enough rep to comment just yet, so based on the information given I will do my best to answer.
It sounds like your rectangle and position of your menu item are not being updated together. Let's say you have a position of (0,0) and rectangle of (0, 0, 100, 100) - (x, y, width, height), so the rectangle extends from (0, 0) to (100, 100). If you update the position to (100, 100), for example, you also have to update the rectangle's x and y coords to (100, 100), thus the rectangle now extends from (100, 100) to (200, 200).
I don't have enough information from your code samples to tell you what's going wrong where, but from what I can tell that's what is happening.
On another note.. if you are passing in the camera's transformation matrix to SpriteBatch.Begin(...) and you are also making draw calls to your HUD elements within that SpriteBatch.Begin...End section, you should consider making another SpriteBatch.Begin...End section without the camera's transformation matrix being passed in. This way your HUD elements' positions will be in screen coordinates compared to world coordinates, and you won't run into issues like this. The HUD elements typically never change positions, unless of course you allow them to, but you will still be working in screen coordinates compared to world coordinates.
Example: (Keep in mind that I don't recall what the parameters are exactly for SpriteBatch.Begin(...) but this should help)
//Main Draw function
public void Draw() {
//World Coordinates
spriteBatch.Begin(cameraTransformationMatrix);
//Draw all world coordinate objects
//Player, Enemies, Walls, etc.
spriteBatch.End();
//Screen Coordinates
spriteBatch.Begin();
//Draw all screen coordinate objects
//Menus, In-Game HUD elements, etc.
spriteBatch.End();
}
Right now I'm using XNA 4.0 with Windows Phone Developer Tools to create a textured cube using a predefined quad class on MSDN.
The front/back/left/right faces of the cube will draw fine (for every cube that I make), however the top and bottom faces won't render. The rasterizer state's cull mode is set to none and the quad that represents the top face exists, and it seems as if it would draw, but for some reason it won't.
Is there a problem with my code, or is this not happening for some other reason?
Here's the code:
Game1.cs: http://pastebin.com/RHU7jNXA
Quad.cs & Cube.cs: http://pastebin.com/P9gz5q4C
It's because your top and bottom faces have a height to them. They should have 0 height.
Here you are passing in a value as height:
Faces[4] = new Quad(topFaceOrigin, Vector3.Normalize(Vector3.Down), Up, Size, Size);
And then here in Quad constructor it's being used to give incorrect LowerLeft & LowerRight values:
LowerLeft = UpperLeft - (Up * height);
LowerRight = UpperRight - (Up * height);
I would recommend changing how you create all your quads; each face really should have different parameters. Right now all your faces are passing in practically the same stuff.