I'm trying to implement hand gesture recognition for Oculus Quest with Unity and the Unity Oculus integration package.
I've read the "Hand Tracking in Unity" documentation on the Oculus developer website, but they only talk about getting the current pinch of the fingers, which is not what I want:
https://developer.oculus.com/documentation/unity/unity-handtracking/
I thought about getting fingers flexion for each finger (with a value between 0 and 1 for example), and then training a k-NN model with the 5 features to then be able to recognize the nearest gesture. But I've been searching for hours and didn't find anything about getting finger position, the only thing I found is getting the pinch.
By looking in the OVRSkeleton.cs file (from the Oculus Integration package), I've been able to get the current Transform for each bone (so the position as a vector and the rotation as a quaternion), but I don't really know how to calculate or get an estimate for the finger flexion with that (or anything useful to perform gesture recognition)
OVRSkeleton skeleton = GetComponent<OVRSkeleton>();
skeleton.Bones[(int) OVRPlugin.BoneId.Hand_Index1].Transform.position
skeleton.Bones[(int) OVRPlugin.BoneId.Hand_Index1].Transform.rotation
The list of bones IDs is in the "Hand Tracking in Unity" documentation page.
In fact, what I want to implement seems to look exactly like this package:
https://assetstore.unity.com/packages/tools/integration/vr-hand-gesture-recognizer-oculus-quest-hand-tracking-168685
Any help, ideas or comments about how to calculate fingers flexion, or any other solution to implement gesture recognition would be greatly appreciated!
Thanks
A few things/links I've explored so far:
https://www.reddit.com/r/OculusQuest/comments/elrn7a/unity_hand_tracking_and_different_gestures/
https://forums.oculusvr.com/developer/discussion/89615/detect-custom-hand-gestures
https://github.com/jorgejgnz/HandTrackingGestureRecorder is something im trying currently, the docs dont say it has a depenedency, but apparently it does, and i dont have it working yet.
you can access the bone rotations at runtime and compare those to a recorded gesture. i think its closer to "template matching" than machine learning, measuring the error between two poses.
I am working on a 2.5D side scroller/platformer project just kind of as a learning experience. I imported a robot model that came with premade animations. I have 2 questions regarding setting him up for movement and his animations. First, I set up movement for x and y, but do I only need to do x? Right now if I hit the UP arrow, the character appears to jump, but feel like that is probably bad form. Am I correct in assuming I should do just the x value for movement and create a separate method for jumping? My second question is with the animation controller. I set up all my states and set my transitions, but I'm not sure what to do from there. What do I do to actually set up the transitions, if anything, and also I could use some information on setting up the animations on the programming side. I have some very slight general knowledge of it, but not much and I'm having trouble finding good solid tutorials for these specific questions.
Thanks for the help!
Yes, I would recommend doing separate controllers and script files for side to side moving and jumping. That way if you ever need to change one of those behaviours you have a specialized place to do it. Also, perhaps you have an enemy that moves side to side but doesn't jump. Or perhaps something that jumps but doesn't move. You could just drag the correct behaviour onto the game object.
as for animations, the unity learn tutorials are pretty helpful getting you started on animations and very basic state transitions
I'm making a simple 2D game in Unity. The gameobject's position is different from Unity editor to my iPhone. For example, if I want to place a gameobject in the center, then I need to place it a little further up before it is in the center on my iPhone. And i'm using iPhone 5 resolution in unity. Hope you guys can help or explain why this happen.
Check the offsets of parents of the game object. It's possible that the outtermost object isn't set to be at 0,0.
Otherwise we're probably going to need to have more information to help you. Screenshots or code snippets or something.
Anyone know how to use Goblin XNA to implement ball control in the same way as one might find in the board game labyrinth?
There don't appear to be any tutorials or information at all regarding how to do this, despite there being a demo video displaying just such a thing.
I've setup the environment and gravity and added the ground and a sphere. I use WorldTransformation.Decompose to extract the current orientation of the board. I know the next step will be either ApplyLinearVelocity or AddForce to the sphere based on the current board orientation, but I don't know how to constantly apply these methods to the ball so that the ball is moving in response to the movement of the ball. Adding code to the Draw or Update methods only executes the code a single time. Anyone familiar with Goblin XNA at all and able to help?
As far as I can see in Goblin XNA the Update and Draw methods are called the same way as standard XNA games. Can you give more specific information? source code perhaps?
I'm currently trying to implement a marble maze game for a WM 5.0 device and have been struggling with developing a working prototype. The prototype would need the user to control the ball using the directional keys and display realistic acceleration and friction.
I was wondering if anyone has experience with this and can give me some advice or point me in the right direction of what is essential and the best way to go around doing such a thing.
Thanks in advance.
Frank.
When reading your answer I didn't get the feeling you are looking for a game framework, but more: how can I easily model a ball with acceleration and friction.
For this you don't need a full fledged physics framework since it is relatively simple to do:
First create a timer which fires 30 times a second, and in the timer callback do the following:
Draw the maze background
Draw a ball at ballX, ballY (both floating point variables)
Add ballSpdX to ballX and add ballSpdY to ballY (the speed)
Now check the keys...
if the directional key is left, then subtract a small amount of ballSpdX
if the directional key is topleft, then subtract a small amount of ballSpdX and ballSpdY
etc
For collision do the following:
first move the ball in the horizontal direction. Then check the collisions with the walls. If a collision has been detected, then move the ball back to its previous positions and reverse the speed: ballSpdX = -ballSpdX
move the ball in the vertical direction. Then check the collisions with the walls. If a collision has been detected, then move the ball back to its previous positions and reverse the speed: ballSpdY = -ballSpdY
by handling the vertical and horizontal movement separately, the collision is much easier since you know which side the ball needs to bounce to.
last nu not least friction, friction is just doing this every frame: ballSpdX *= friction;
Where friction is something like 0.99. This makes sure the speed of the ball get's smaller every frame due to friction;
Hope this helped
I would recommend checking out XNA Studio 3, it has built in support for PC, Xbox 360 and mobile devices, and it's an official & free spin-off of Visual Studio from Microsoft.
http://creators.xna.com/en-US/
http://blogs.msdn.com/xna/
If you search around, people have written tutorials using physics (velocity on this one)
http://www.xnamachine.com/2007/12/fun-with-very-basic-physics.html
Try XFlib. It is in c++, but most cool things for the mobile have to be in c++, unfortunately. The site has some very cool free games. You can also see the source of most of the game too. Many have the physics you want.
Unfortunately, XNA doesn't support the windows mobile platform. However, as it seems that you're not having a problem with the technical issue of drawing on the WM device, but with the logic required to implement physics based movement, then it's not a bad idea to consider XNA to prototype the physics and movement code.
Check out some of the educational topics at creators.xna.com, and also "gamedev.net"
If you are at a loss, there's no mistake in trying a "lighter" tool for prototype. I would try Torque Game Builder - it spits out XNA, although maybe not meant for your platform.
At the Samples of the Windows Mobile SDK (check out the WM 6.0 SDK too), there are a couple of game applications. One of them is a simple puzzle game; not much, but it is a starting point.
The use of physics in game development is not specific for Windows Mobile. You can find a huge literature about this subject. This comes up in my mind now. If you are serious about game development, in any platform, you should do a little research first.
I dont know if this may help but i saw a Marble application for the Android platform on google code. Check it out here, it may throw some insight on the actual logic of the game.
The code is open sourced and written in java (using the android sdk) put nevertheless it may be useful. Also to better understand the code checkout the documentation for the SensorsManager, SensorEvent etc here
I wouldn't recommend using the same architecture as this application thou.