I'm working on a project where I'm trying to implement Q-learning in C#(Unity specifically). I have a car which needs to drive along a grid path, like shown:
Example - Where Green is goal(+reward), gray and off-grid are obstacles(-reward) and blue is the car. I've been looking at a few examples/guides on how to implement this (pseudo-code mostly), but they seem to know a pre-defined grid to work(A reward grid), which I'm unsure how to implement in my scenario.
The idea is that my car have three states & actions:
States: Driving in left lane, driving in center lane, driving in right lane
Actions: Move one up and left, move one up and right, move one up
I'm trying to follow this: http://mnemstudio.org/path-finding-q-learning-tutorial.htm and adapt that into my system - but I fail to see how to do the adaption. Wouldn't this require my Q-matrix to be much`much larger, as each cell would be an individual state? Lets say my track is 16 cells long, with 3 lanes, would this mean I should have a 48 states * 3 actions grid?
Edit: To simplify my post, I thin what I'm struggling with is how to connect my environment and my Q-matrix, so I can modify my q-matrix values depending on the behaviour of my agent in my kind of system.
Typically, in grid-based environments like your car example, each cell corresponds with a different state. So, your environment may have the 3 actions you described (move one up and left, move one up and right, move one up), but definitively doesn't have three states.
As you point in the last part of the question, if your track is 16 cells long and 3 lanes, your environment should be modelled with 16 * 3 = 48 states. Therefore, your Q-matrix size should be 48 x 3.
With this setup, you can easily map your agent state (car position in the track) with the Q-matrix.
Related
Part 1: Forgive me if the question itself is unclear. I am learning how to use Unity and script in C#, and I want to know if there's a way to apply a gradient of color (or an image) utilizing the game objects that already exist as the places where the gradient will show up.
Say I have a group of these circles that randomly grow and change size during the game run.
Example image of circles
I am not sure of the correct terminology, but a couple of words come to mind, i.e. shader/mask. My goal is to display the gradient/image only within where the game objects exist. So instead of white circles, it's circles display parts of the one singular image/gradient.
Part 2:
To take a it a step further, I'd like to know how to have the gradient continuously run through its spectrum so one can see the colors shift across the circles.
Again, still very new to this kind of stuff, but would anyone know what steps I would need to take to get there.
Thanks!
I'm having some issues with items of clothing on an avatar as you can see in the image below
Most would suggest that i simply pull the trousers away from the body within the modelling software which is fine until it is combined with other items of clothing
I could now pull the shirt away from the trousers but I have many different items of clothing and to check/adjust each of them does not seem like the way to go.
It would be better to try and apply some sort of layering system.
Can anyone advise me on how i may be able to achieve such a system or a how i can achieve the results I am aiming for?
I have already looked into trying to do this using shaders but had no luck achieving the result I'm trying to get (but admittedly I am no Shaderlab/CG expert)
I have also tried separating the avatar into different pieces and replacing them with the clothing meshes but this still leaves the problem of the clothing being visible through each other.
I would like to preferably have some sort of system where i can layer each item of clothing so that no matter what they display on top of each other in the correct order without any overlaps.
If you really want to use layering you can use Unity's Render Layers which are actually made for that.
Go to Layers -> Edit Layers
Add the Layers you want (the order doesn't matter)
Asign the Layers to the objects you want to be rendered by a ceratin camera.
as an example I added a trousers cube with layer trousers, a pillover cube with layer pullover and a OnTop cube with layer OnTop
Create a new Empty GameObject and call it e.g. CameraParent. This will contain all your cameras and you should move this one instead of the camera you moved so far.
In case this is a VR/AR app you might have to attach a Camera component to this parent object, make it the main camera but make it not render anything
Create e.g. 4 Child objects and give them a Camera component. Also make sure only one Camera is tagged as MainCamera and only one has an AudioListener in your Scene. I used 4 cameras as example to have a TrousersCamera a PulloverCamera an OnTopCamera and a BackgroundCamera.
Now the setup for the rendering
BackgroundCamera
Here you render anything else like you did before
ClearFlags: e.g. SkyBox
Culling Mask: select all Layers except trousers, pullover and OnTop
Depth: -1
TrousersCamera
For rendering the trousers (on top of anything else)
ClearFlags: Depth only
Culling Mask: select only trousers
Depth: 0 (you see a bigger value is rendered on top of a lower value)
PulloverCamera
For rendering the pullover on top of the trousers
ClearFlags: Depth only
Culling Mask: select only pullover
Depth: 1
OnTopCamera
For rendering something else on top of everything
ClearFlags: Depth only
Culling Mask: select only OnTop
Depth: 2
As result you can see that though the red cube (OnTop cube) is really far behind the rest and the blue cube (Pullover cube) in the middle, the red one will allways be rendered on top, than the blue, than the brown (trousers cube) and finally the background.
So I created this window editor in WPF that helps me create Forms quickly. Now, one feature I've worked on was create a guideline tool. At its core it's just creates lines to help keep my UI elements organized on the screen. I will show you an example. The long black lines are the guidelines I spoke about earlier.
Now, I noticed that in a lot of art programs (i.e Photoshop) and popular IDEs that implement Forms that they have a "snap-to" feature where a UI element will snap to a line UI or to another UI element in order to maintain alignment. Something like this:
I already have the guidelines showing up in my editor. Now, what I would like help understanding is, how would I go about implementing the "snap to" feature? I'n not asking for code, just a breakdown (a visual breakdown will be most welcomed).
These are my questions:
How does an object know if one of its edges (top, bottom, left, right) touched a line?
How would I know how to unsnap the UI element if the user keeps moving the mouse past the guideline?
If I have (say) 10 lines how do I make sure that the object attaches to the nearest line(s)?
UPDATE
When an object moves or is resized, keep track of its actual size/location relative to the mouse, and separately keep track of a snapped version of the same information. If a given actual edge is within some arbitrary distance of a line -- say 4 pixels (arbitrary WPF units, really). If it's within that distance, set it to the value for the line it's close to. You still have the actual mouse-relative values as well, so you know to unsnap it if the the user keeps on dragging it and it leaves that 4-unit zone.
When an object is being resized, at most two edges of the bounding box will be changing position (assuming you can drag corners as well as edges). When you're moving an object, all four edges of the bounding box will move.
So you need to keep track of which edges are moving, and only do snap-line proximity testing on those edges. When you're moving an object, snapping the left or top edge to a line is easy. That's just the position of the object. But if you snap the right or top edge to a line, you're setting
snappedPos.X = nearestVerticalSnapLine.X - draggedObject.Width;
or
snappedPos.Y = nearestHorizontalSnapLine.Y - draggedObject.Height;
You may also have cases where opposite edges will both be in proximity to lines: Say you're dragging a seven-unit square across a ten-unit grid. When it's inside a grid box, all four sides will be in proximity to a grid line. Which wins? The closer one.
Locating the snap lines is easy -- %.
using a file I want to create a map and I am wondering about the best approach doing so.
Actually I searched the forum but I only found map generation algorithms that randomly creates maps.
Let's look at a minimal example.
e.g. Ihave a file containing
0110
1001
1000
0000
Every 0 shall be water and every 1 shall be earth.
I would handle this by simply havin two different bitmaps and loading them at the right coordinates. That'd be simple.
But let's guess we have a 1000*1000 big map and there is only enough space for 16*16 tiles per frame. Then I'd get the current position and would build the map around it.
Assuming we can only display 3*3 tiles, using the minimal example and being at position (2,2) where x and y is element 1..4 so what the user could see at this time would be:
011
100
100
Solution
I thought about using a text file, where a line represents the x-coordinate direction and
a column represents the y-coordinate direction. The whole file is being loaded at the beginning of the program. This shouldn't use too much ram assuming 1 tile needs 1 byte, what should be enough.
For redrawing the map when the user is moving, I'd get the moving direction and slide the current Bitmap for the height/width of a tile in the opposite direction and only look up the bitmaps for the new blank spaces. So I only need to look up the tile information for m+n-1 (where m is the amount of displayed tiles in y and n in x direction) tiles (max case if moving diagonal) instead of loading m*n tiles everytime the user moves.
Example
I created an example to make the above given example more easily to understand.
this is the whole map:
We can only display 3*3 tiles and the user is at position (2,2) so what we'd actually see is:
now he is moving towards the bottom right corner:
and the black framed section is being move to the opposite direction, so that we get:
now the blank tiles (black framed white areas) have to be looked up and teh final result will be:
Question
is this a good way of building a map? Or are there much faster functions, maybe already implemented in the microsoft xna-gamestudio package ?
I would pre-fetch 1-2 tiles range outside the screen view, so that you won't have weird pop-up as the player move.
But if your game is a top-down tile game, this solution is quite conservative. In most hardware today, you could create a very big range around the player without problem. Just look at the number of block Minecraft can process and display. Since you are reusing the same texture, you just load the asset once and reuse them in a tile, which would probably an object with very little memory footprint.
Have you tried implementing it yet?
I am trying to write a card game, where a player is able to stack cards. E.g. Ace, Two, Three.
I would like to visualize a stack of cards, where the Ace card is partially covered by the Two card, and the Two card is partially covered by the Three card. The Three card is completely visible.
Easy, I thought. I make a user control where I add my cards: Controls.Add(ace); Controls.Add(two); etc.
Then I need something that is able to lay-out my Controls, so I wrote my custom LayoutEngine (derives from LayoutEngine). My first test does nothing more then shift the control 50 pixels.
After running the solution I noticed that the Z-ordering was wrong. Instead of the Three card being on top, the Ace card was on top looking like this:
Ace Card > Two card > Three card where:
Ace Card is on top
Two Card is under the Ace Card
Three Card is under the Two Card.
So I started looking for a way to change the Z order in WinForms and found out that it is simply "not available". Like.. Huh?!
The alternative (provided by MS) is that the Z order can be altered, by setting the ChildIndex for the Controls. Jikes, that means that poking around in a list, change the behaviour of my application. Way to go MS...
Anyway, I tried all kind of things, but it seems impossible to write a layout engine that does the trick.
I've google-d all day, and found nothing useful. I am no GUI expert, so I run stuck on this lame issue. Who can help me out?
Much appreciated!
Bas
Your best bet is to avoid the use of controls entirely. They will A) result in poor(er) performance and B) complicate hit testing/drawing.
Simply create objects to represent the state of the table (I use a CardContainer object) and use Graphics.DrawImage to draw all of the cards where they lie during the paint event. You can use a single control for the entire table if you need to also add other UI elements.
This will also make animating card movement simpler should you decide to add animation.
Updated
I meant to expand this answer but was called away and simply posted what I had. Here are some details you may find useful. I created a "solitaire game engine". The engine hosts one solitaire game at a time (klondike, spider, strategy, etc.). It tracks statistics for each game and allows both playing and editing of the individual games. The games are IronPython scripts which makes adding new games relatively easy.
My CardContainer is an object that holds zero or more cards.
It has a LieDirection (None, Up, Down, Left, Right) which determains how its cards are laid out.
It has a MaximumDepth that clamps the number of cards drawn in the LieDirection. This is handy for games like Klondike where you only want to show the top 3 cards of the waste.
It has properties for spacing the cards. There are separate spacing values for cards that are face up and face down. It can auto-pack cards into an area defined by MaximumLength. And it has an 'extra pad' value, one for each card--whether there is a card at that index or not. The latter is used during a simulated mouse hover to 'uncover' the card pointed to so that the user can clearly see a card that might be obscured by cards on top of it. This is accomplished by setting the 'extra pad' of the card on top of the hover card. This could have been simplified by having a "hover card" and "hover spacing" property, but having extra padding per-card allows for odd kinds of solitaire games that highlight a particular 'row' in the tableau piles with spacing.
It has a HitTest method to return a Card from a given X,Y location.
All of that means that the Card object has no notion of where it is drawn on the table. I have a complex animation system and so a card's location ultimately comes from the animation engine. If a card is not currently animating, the animation system gets its location from its container.
Note that the card's location referred to above is strictly for drawing. All cards are always attached to exactly one CardContainer and are simply moved from one to another. There is one 'special' container called the Deck which initially contains every card. It is positioned off of the table initially. A container has a Visible property. Animations play only if moving a card from a Visible container to another Visible container. This allows you to move cards around without animation when necessary and cards can "fly out" to/from containers positioned off the table.
The engine also has a rudamentary layout system for positioning CardContainers relative to each other. One very handy thing I did was to use a card-size-relative coordinate system. The 'width' of the table is exactly 11 card widths. No matter how big the user sizes the table, the width is always 11 card widths. This means that the card sizes (as viewed by the user) grow and shrink. The height is variable, but is determained by a fixed card-sized ratio (determained from the card bitmaps). If you give a CardContainer an X value of 1.0, that means it will be located one card-width from the left of the table. The values are floating points so you can specify 1/2 a card-width with 0.5. This makes it very easy to position elements in the script without having to worry about screen coordinates. No matter how the user alters the size of the screen, your game will be laid out exactly the same way.
The engine also has unlimited undo and redo. This means that not only do card moves (from one container to another) have to be recorded, but all property changes are recorded as well (both card and container properties). Undo and redo can be difficult to implement if not planned for from the start. The scripts have access to a Game.LogVariableChange method so that they can alter the value of a global variable through the recording mechanism. This is necessary for something like Klondike's "three redeals" feature. The script has to store the number of redeals used, but if the user undid a redeal, that variable's value change has to be undone too.
This works very well for Solitaire, but could work for just about any kind of card game. Obviously you don't have to go and implement all of this your first time out. I present the information just to give you some ideas.