How can I tell whether a person is facing a Kinect or showing it his/her back?
I am using the Microsoft Kinect SDK v1.7.
The Microsoft Kinect SDK does not track the back of users. It is unable to track a full body rotation, but only a more or less forward facing user.
Now granted, it might get "confused" and be able to track your skeleton when you're facing it with your back, but even then the skeleton will be aligned as if you were facing forward. If it does track you could potentially apply a heuristic that says "are my wrists further from the sensor than my hips?" or "how is the shoulder, elbow, wrist angle oriented", but it would all be inaccurate at best.
When the user stay without any movement it is impossible.But if user is walking the scenario can be solved like this:
Regarding to the Skeleton Coordinates in SDK 1.7, the Z coordinates is the distance from the user to the Kinect, So when user walk toward the camera Z is decreased and when user is going far from the camera the Z is increased.
This scenario is only useful when your user is walking normal.
Related
I'm trying to implement hand gesture recognition for Oculus Quest with Unity and the Unity Oculus integration package.
I've read the "Hand Tracking in Unity" documentation on the Oculus developer website, but they only talk about getting the current pinch of the fingers, which is not what I want:
https://developer.oculus.com/documentation/unity/unity-handtracking/
I thought about getting fingers flexion for each finger (with a value between 0 and 1 for example), and then training a k-NN model with the 5 features to then be able to recognize the nearest gesture. But I've been searching for hours and didn't find anything about getting finger position, the only thing I found is getting the pinch.
By looking in the OVRSkeleton.cs file (from the Oculus Integration package), I've been able to get the current Transform for each bone (so the position as a vector and the rotation as a quaternion), but I don't really know how to calculate or get an estimate for the finger flexion with that (or anything useful to perform gesture recognition)
OVRSkeleton skeleton = GetComponent<OVRSkeleton>();
skeleton.Bones[(int) OVRPlugin.BoneId.Hand_Index1].Transform.position
skeleton.Bones[(int) OVRPlugin.BoneId.Hand_Index1].Transform.rotation
The list of bones IDs is in the "Hand Tracking in Unity" documentation page.
In fact, what I want to implement seems to look exactly like this package:
https://assetstore.unity.com/packages/tools/integration/vr-hand-gesture-recognizer-oculus-quest-hand-tracking-168685
Any help, ideas or comments about how to calculate fingers flexion, or any other solution to implement gesture recognition would be greatly appreciated!
Thanks
A few things/links I've explored so far:
https://www.reddit.com/r/OculusQuest/comments/elrn7a/unity_hand_tracking_and_different_gestures/
https://forums.oculusvr.com/developer/discussion/89615/detect-custom-hand-gestures
https://github.com/jorgejgnz/HandTrackingGestureRecorder is something im trying currently, the docs dont say it has a depenedency, but apparently it does, and i dont have it working yet.
you can access the bone rotations at runtime and compare those to a recorded gesture. i think its closer to "template matching" than machine learning, measuring the error between two poses.
I am working on a Unity leap motion desktop project using latest versions of Orion and unity prefabs package. I have created a simple scene pretty much identical to the one in Desktop Demo scene. (pair of capsule hands and a simple object you can poke and move around)
https://developer.leapmotion.com/documentation/csharp/devguide/Leap_Coordinate_Mapping.html
This article covers all of the issues I am currently facing but so far, I was unable to implement these solutions in my project.
When moving hands from the maximum range of the camera for example left to right or any other direction, this only translates to a portion of available screen space, in other words, a user will never be able to reach out to the edges of a screen with their hands. From my understanding, the tracking info provided in millimetres by the camera is somehow translated into units that Unity can understand and process. I want to change that scale.
From the article, "You also have to decide how to scale the Leap Motion coordinates to suit your application (i.e. how many pixels per millimetre in a 2D application). The greater the scale factor, the more affect a small physical movement will have." - This is exactly what I want to do in Unity.
Additionally, even being able to what I think was a successful attempt at normalisation of coordinates using the InteractionBox, I am unsure what to do with the results. How or rather, where do I pass these values so that it will display the hands in an updated position.
I am trying to make a app in c# that will detect / classify 3 poses of human body which are standing, sitting and lying. I can correctly detect / classify 2 of them (sitting and standing) with skeleton tracking. When it comes to lying on the floor, Kinect seems to not be able to track skeleton of a human body.
Does anyone have any experiences with skeleton tracking in lying position? As soon as I lye down, I am loosing joints positions. Is this task impossible? Thank you.
The Windows SDK for Kinect V1 is not great at recognizing people lying down.
V2 improves a lot on this, I would recommend you to participate in the beta or wait for V2.
One possible solution with V1, is to place a second camera vertically on the floor, use seated mode - so it detects on motion - and see if you find a person with that camera and not on the other, then it is likely lying down.
I have not tested this solution, but in theory it can work - try to set it up yourself and see if it fits your scenario needs.
I'm now detecting all the skeleton in a wpf application, I want to know how to detect the fingers to appear with the skeleton? I'm using microsoft Kinect for windows sdk ver 1.5
Many thanks
The Kinect unfortunately is not sensitive enough to recognize fingers so the library will not provide that as part of the skeleton. Maybe the Kinect 2.0 rumored to come out with the Xbox 720 will be able to provide that level of detail.
Candescent NUI might be what you're looking for. As OpenUserX03 said, however, the Kinect isn't ideal for this task. Perhabs you should have a look at the coming-up LEAP technology, which specializes in finger detection.
The cameras on the Kinect are not meant to be able to do joint tracking for the hands to that level of detail. Tracking the individual fingers is possible but wont be very reliable. To represent a players hand in the skeleton, you can check if the players hand is opened or closed. A possible way to see if the hand is open or closed would be to do pixel checks in an area surrounding the hand. This way with some tuning you could calculate how much of that area is the hand (using the depth and color stream) and how much is not. For example: If 40% of that area is the same depth as the hand joint, the hand is closed in a fist. If 70% of that area is the same depth as the hand joint, the hand is open. Then you could possibly use the angle of the elbow and wrist joint to be able to represent a closed or open hand at that angle on the skeleton.
I'm working with the C# managed wiimote library for a little fun project I'm working on, But I'm having trouble finding a good tutorial on how to calculate how far the wiimote is from the monitor (i.e sensor bar). I want to create a zoom effect where an object will grow or shrink based on how far back you move the wiimote from the screen.
Can anyone help me with this?
The system is not calibrated, so it isn't able to tell you the actual distance. However it can tell you the relative distance. The IR sensor on the remote works by telling you the size and location of up the IR light sources on the sensor bar. When the remote moves farther away, the lights get smaller and closer together; when the remote gets closer, the lights get larger and farther away from each other. I would use the distance between the lights, as the size of the dots only goes from 0-15.
I recommend Brian Peek's Wii library: http://wiimotelib.codeplex.com