Kinect's SkeletonStream to feedback user undetected C# - c#

I am trying to get my Kinect's SkeletonStream to feedback a data that tells me that nobody is detected. I can get feed if my skeleton is detected but I am unable to get any notification if there is nobody there. Is there a way to get the kinect to tell me if no skeletons are picked up?
private void kinect_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
using (SkeletonFrame frame = e.OpenSkeletonFrame())
{
if (frame == null)
{
return;
}
frame.GetSkeletons(ref allSkeletons);
if (allSkeletons.All(s => s.TrackingState == SkeletonTrackingState.NotTracked))
return;
foreach (var skeleton in allSkeletons)
{
if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
{
continue;
}
if (skeleton.TrackingState == SkeletonTrackingState.NotTracked)
{
}
foreach (Joint joint in skeleton.Joints)
{
if (joint.TrackingState != JointTrackingState.Tracked)
continue;
if (joint.JointType == JointType.HipCenter)
{
hipCenter = joint.Position;
AdvanceFunction();
}
}
sdm.Draw(frame.GetSkeletons(), false);
}
}
}

There is nothing that will simply tell you that no skeletons are currently being tracked. You will need to look through the skeleton frames to determine if there are any users.
Your foreach loop steps through all the skeletons...
foreach (var skeleton in allSkeletons)
{
if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
{
continue;
}
....
}
In the first if statement -- if the current skeleton is not being actively tracked then the loop will move on to the next skeleton. You'll want to add a flag if any skeletons have been found. For example, you could do something like...
bool hasActivePlayer = false;
foreach (var skeleton in allSkeletons)
{
if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
{
continue;
}
hasActivePlayer = true;
....
}
if (hasActivePlayer == false)
{
// you aren't tracking anyone, deal with it
}
You may also be interested in checking SkeletonTrackingState.PositionOnly. In this case the Kinect knows someone is there, but it is not actively tracking their skeleton. You can update the ckeck in the foreach loop if you want to look for it as well.

Related

Take the full path waypoints of a railway

I'm developing a Train Valley game, just like 2D. The construction of the railway is different. I have a grid and I can take a rail and put it somewhere in the grid and so the railway is built. The main problem I am facing is that I do not know how to create the way for the train to follow .... each sine has a point, this point I add in a Vector3 type list, but if I jump over a cell and put there sine , then go back to the skipped cell and put the rail there, then the list will no longer be sorted.
I'm attaching code, but it's a lot. In general, if anyone has any idea how to build this path, the train will run correctly by all means.
void Start()
{
foreach (TrainCell cell in Grid)
cell.gameObject.AddComponent<TrainCellMouseEvent>().MouseEvent += OnCellClickEvent;
MouseEventProccessor.Instance.captureMouseMouveEvents = true;
StartCoroutine(SpawnStations());
}
private void OnCellClickEvent(TrainCell target, MouseEventType type)
{
if (type == MouseEventType.CLICK)
{
if (canDestroy)
{
if ((int)target.CurrentChildIndex != 0)
{
target.CurrentChild.GetComponent<PolygonCollider2D>().enabled = false;
target.setCurrentChildIndex(0);
Instantiate(BoomFX, target.transform.position, Quaternion.identity);
target.used = false;
WayPoints.Remove(target.transform.position);
if (BuildDone != null)
BuildDone(price, false);
}
foreach (TrainCell cell in Grid)
{
if (!cell.underBuilding)
{
if (cell.CurrentChildIndex == 0 && cell.used)
cell.used = false;
}
}
return;
}
if (rail.SelectedRail == null || target.used || outOfGridBounds)
return;
int railIndex = (int)rail.SelectedRail.GetComponent<ObjectSequence>().CurrentChildIndex;
target.setCurrentChildIndex(railIndex + 1);
target.CurrentChild.GetComponent<PolygonCollider2D>().enabled = true;
MouseEventProccessor.Instance.captureMouseMouveEvents = true;
SpriteRenderer render = target.CurrentChild.GetComponent<SpriteRenderer>();
render.color = Vector4.one;
target.used = true;
if (BuildDone != null)
BuildDone(price, true);
if (target.CurrentChild.transform.childCount == 0)
WayPoints.Add(target.transform.position);
else
{
for (int i = 0; i < target.CurrentChild.transform.childCount; i++)
WayPoints.Add(target.CurrentChild.transform.GetChild(i).transform.position);
}
}
else if (type == MouseEventType.OVER)
{
if (mainLevel.isHammer)
{
if (target.CurrentChildIndex != 0)
target.CurrentChild.GetComponent<SpriteRenderer>().color = Color.red;
return;
}
if (rail.SelectedRail == null || target.used)
return;
foreach (TrainCell cell in Grid)
{
if (cell.CurrentChildIndex != 0)
{
if (cell != target)
{
if (cell.CurrentChild.GetComponent<SpriteRenderer>().color != Color.white && !cell.used && !cell.underBuilding)
{
if (cell.GetComponentInChildren<ObjectSequenceNumberDisplay>() != null)
Destroy(GetComponentInChildren<ObjectSequenceNumberDisplay>().gameObject);
cell.setCurrentChildIndex(0);
}
}
}
}
int railIndex = (int)rail.SelectedRail.GetComponent<ObjectSequence>().CurrentChildIndex;
target.setCurrentChildIndex(railIndex + 1);
RenderColor(target);
}
else if (type == MouseEventType.EXIT)
{
if (target.CurrentChildIndex != 0)
target.CurrentChild.GetComponent<SpriteRenderer>().color = Color.white;
if (shapePrice != null)
Destroy(shapePrice.gameObject);
if (target.used)
return;
target.setCurrentChildIndex(0);
}
}
As far as I suppose based on your code I'll try to answer your question based on the following statements:
You have a station where you generate the train
You define a direction for the train to follow from the station (left, right, up, down) depending on the railway design
Having that statements I propose the following solution:
Create a method that takes the station point as the origin and the starting direction from that point. Having that you start following the railway, first on that direction and after that you search in 4 directions for more rails, adding all of them to the Vector3 array. This way you will have the array sorted.
Example:
private void SortedRailway(Vector2 origin, Vector2? direction = null, Vector2? lastDirection = null){
if(direction.HasValue){
if(origin + direction is TrainCell){
WayPoints.add(origin + direction);
SortedRailway(origin + direction, null, direction);
}
//(else will be used if direction (just used for the initial one) is wrong, so error there)
}else{
if(origin + Vector2.up is TrainCell && Vector2.up != lastDirection){
WayPoints.add(origin + Vector2.up);
SortedRailway(origin + Vector2.up, null, Vector2.down); //lastDirection is always the opposite to the matching one
}
//... (Repeat for all 4 directions. If none fits, then we asume is the end of the railway)
}
}
Hope this helps you!

Understand Kinect Facetracker.track() method

I'm Using Kinect v1 with SDK 1.8. I need to pass my data to faceTracker.Track() method.
But in constructor of track() is asking Skeleton parameter at the end.
But cannot take directly skeleton data to skeleton object. It provides as an data array.
How can I pass data to this faceTracker.Track() method as the track method requested?
The following is the way I tried.
using (SkeletonFrame skeletonFrame = allFramesReadyEventArgs.OpenSkeletonFrame())
{
if (skeletonFrame != null && this.skeletonData != null) // check that a frame is available
{
skeletonData = new Skeleton [newSensor.SkeletonStream.FrameSkeletonArrayLength];
skeletonFrame.CopySkeletonDataTo(this.skeletonData); // get the skeletal information in this frame
this.skeletonData = new Skeleton[this.newSensor.SkeletonStream.FrameSkeletonArrayLength];
}
}
track method
if (faceTracker == null)
{
try
{
faceTracker = new FaceTracker(sensorChooser.Kinect);
}
catch (InvalidOperationException)
{
//write something
}
if (faceTracker != null)
{
FaceTrackFrame frame = faceTracker.Track(
sensorChooser.Kinect.ColorStream.Format,
colorImageData,
sensorChooser.Kinect.DepthStream.Format,
colorPixels,
skeletonData);
}
}
i defined Skeleton [] skeletonData ;as this
and follwing is the track method given in Kinect SDK
public FaceTrackFrame Track(
ColorImageFormat colorImageFormat,
byte[] colorImage,
DepthImageFormat depthImageFormat,
short[] depthImage,
Skeleton skeletonOfInterest)
{
return this.Track(colorImageFormat, colorImage, depthImageFormat, depthImage, skeletonOfInterest, Rect.Empty);
}
foreach (Skeleton skeleton in this.skeletonData)
{
if (skeleton.TrackingState == SkeletonTrackingState.Tracked
|| skeleton.TrackingState == SkeletonTrackingState.PositionOnly)
{
// We want keep a record of any skeleton, tracked or untracked.
if (!this.trackedSkeletons.ContainsKey(skeleton.TrackingId))
{
this.trackedSkeletons.Add(skeleton.TrackingId, new SkeletonFaceTracker());
}
// Give each tracker the upated frame.
SkeletonFaceTracker skeletonFaceTracker;
if (this.trackedSkeletons.TryGetValue(skeleton.TrackingId, out skeletonFaceTracker))
{
skeletonFaceTracker.OnFrameReady(this.Kinect, colorImageFormat, colorImage, depthImageFormat, depthImage, skeleton);
skeletonFaceTracker.LastTrackedFrame = skeletonFrame.FrameNumber;
}
}
}
I solved it as above with the help of kinect toolkit 1.8 the project FaceTrackingBasics-WPF application.

.SendMessage Crashes Unity C#

I am using a OverlapSphere to detect all colliders within a certain radius of my object. I then filter out a few I don't care about. With the remaining few, I attempt to send a message to those objects to update their render color. Whenever it sends the message, unity freezes. I tried to do some research and the best thing i could find is that infinite loops can freeze it. But i don't see a potential for that. Here is the code:
Object to send the message:
void sendmyMessage(bool status)
{
Collider[] tiles = Physics.OverlapSphere(gameObject.transform.position, 10);
int i = 0;
while (i < tiles.Length)
{
if(tiles[i].tag == "Tile")
{
//turn light on
if (status == true)
{
tiles[i].SendMessage("Highlight", true);
i++;
}
//turn light off
if (status == false)
{
tiles[i].SendMessage("Highlight", false);
i++;
}
}
}
}
Object Receiving Message:
void Highlight(bool status)
{
//turn light on
if(status == true)
{
gameObject.GetComponent<Renderer>().material.color = new Color(0, 0, 0);
}
//turn light off
if(status == false)
{
gameObject.GetComponent<Renderer>().material.color = new Color(1, 1, 1);
}
}
Any help is much appreciated!
It freezes because of logic if(tiles[i].tag == "Tile") here's your answer. Now imagine that object that you collide with has tag "not a tile"? then the loop never ends.
foreach(var tile in tiles) {
if (tile.tag == "Tile") {
tiles[i].SendMessage("Highlight", status);
}
}
while (i < tiles.Length)
{
if(tiles[i].tag == "Tile")
{
//snip
}
// else - loop forever?
}
Here's your problem. If the tag != "Tile" then you never increment i.

Recognize two skeletons Kinect

I have make a game (pong) using a kinect, I can recognize one skeleton and I can make gestures to control the left stick, but when I try to play with two players donĀ“t recognize the skeleton of other player. This is what I do so far:
private void SensorSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
Skeleton[] skeletons = new Skeleton[0];
using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
{
if (skeletonFrame != null)
{
skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
skeletonFrame.CopySkeletonDataTo(skeletons);
}
if (skeletons.Length != 0)
{
foreach (Skeleton skel in skeletons)
{
if (skel.TrackingState == SkeletonTrackingState.Tracked)
{
this.tracked(skel);
this.trackedLeft(skel);
}
}
}
}
}
public void tracked(Skeleton skeleton)
{
Joint jHandRight = skeleton.Joints[JointType.HandRight];
Joint jHipCenter = skeleton.Joints[JointType.HipCenter];
if ((jHipCenter.Position.Z - jHandRight.Position.Z) > 0.2)
{
//Consider hand raised in front of them
//System.Diagnostics.Debug.WriteLine("Hand: Raised");
//MessageBox.Show("POR FAVORRRRRRRR");
//movement[0] = false;
movement[0] = true;
movement[1] = false;
}
else
{
//Hand is lowered by the users side
//System.Diagnostics.Debug.WriteLine("Hand: Lowered");
//MessageBox.Show("A SERRRRIIIIIOOOOOOOOOOOOOO");
//movement[1] = false;
movement[1] = true;
movement[0] = false;
}
}
Someone could help me.
You need a way to distinguish between skeleton one and two. See Kinect user Detection for how to do this. You can then pass the skeleton for player one and two to your two different methods. I use the player ID because if a skeleton is lost for a frame or two, their ID remains the same
int id1 = 0, id2 = 0;
...
if (skeletons.Length != 0)
{
foreach (Skeleton skel in skeletons)
{
if (skel.TrackingState == SkeletonTrackingState.Tracked)
{
if (skel.TrackingID == id1)
this.tracked(skel);
else if (skel.TrackingID == id2)
this.trackedLeft(skel);
else
{
if (id1 != 0 && id2 == 0)
id2 = skel.TrackingID;
else if (id2 != 0 && id1 == 0)
id1 = skel.TrackingID;
}
}
}
}

Compare two joints position with Kinect

I want to make a Kinect program thanks to, when your left hand X position cross your right shoulder X position, it plays the following slide on PowerPoint. I've already done the program part which draws the skeleton stream, and it's work correctly. For the second part, I've tried this :
private void NextSlide(Skeleton skeleton)
{
Joint leftHand = skeleton.Joints[JointType.HandLeft];
Joint spine = skeleton.Joints[JointType.Spine];
Joint shoulderRight = skeleton.Joints[JointType.ShoulderRight];
bool check = false;
if (leftHand.Position.X == shoulderRight.Position.X && check == false)
{
check = true;
System.Windows.Forms.SendKeys.Send("{RIGHT}");
}
if (leftHand.Position.X == spine.Position.X && check == true)
{
check = false
}
}
Can anyone explain me what is the problem in my code ?
Thanks,
Edit :
I've also tried this :
private void NextSlide(Skeleton skeleton)
{
Joint leftHand = skeleton.Joints[JointType.HandLeft];
Joint spine = skeleton.Joints[JointType.Spine];
Joint shoulderRight = skeleton.Joints[JointType.ShoulderRight];
double lefthandposition = (int)leftHand.Position.X;
double shoulderrightposition = (int)shoulderRight.Position.X;
double spineposition = (int)spine.Position.X;
bool turn = false;
double right = lefthandposition - shoulderrightposition;
bool finish = false;
double ok = lefthandposition - spineposition;
if (right < 0)
{
turn = true;
}
if (ok > 0)
{
finish = true;
}
bool check = false;
if (turn == true && check == false && finish == false)
{
System.Windows.Forms.SendKeys.Send("{RIGHT}");
check = true;
turn = false;
}
if (finish == true && check == true && turn == false)
{
check = false;
finish = false;
}
}
But it doesn't work too :/
The trouble you are running into is that you using == as a check. You will rarely, if ever, run into a situation where two joints are exactly the same value. You need to be using >= or <= for your checks, depending on your needs.
For example, check the JointType.LeftHand >= JointType.RightShoulder, and that JointType.LeftHand <= JointType.Spine, for your position checks.
Another solution could be to use an existing gesture library to accomplish what you want. Here are two of them:
http://kinecttoolbox.codeplex.com/
https://github.com/EvilClosetMonkey/Fizbin.Kinect.Gestures
The Fizbin library's GitHub page explains how to set up and execute on existing gestures: Executing on Gestures. Included in the existing gesture set is a "Swipe Right" and a "Swipe Left" gesture, which execute similar to what you are wanting to do.
The GitHub pages goes into more detail, but a quick rundown would start with setting up the gesture library and a callback:
gestureController = new GestureController();
gestureController.GestureRecognized += OnGestureRecognized;
You'd then initialize the gesture(s) you want to use:
IRelativeGestureSegment[] swipeleftSegments = new IRelativeGestureSegment[3];
swipeleftSegments[0] = new SwipeLeftSegment1();
swipeleftSegments[1] = new SwipeLeftSegment2();
swipeleftSegments[2] = new SwipeLeftSegment3();
gestureController.AddGesture("SwipeLeft", swipeleftSegments);
You can modify the existing gesture components or write your own. The several examples will demonstrate how you can compare joint positions. For example, the first segment in the "SwipeLeft" gesture follows:
public class SwipeLeftSegment1 : IRelativeGestureSegment
{
/// <summary>
/// Checks the gesture.
/// </summary>
/// <param name="skeleton">The skeleton.</param>
/// <returns>GesturePartResult based on if the gesture part has been completed</returns>
public GesturePartResult CheckGesture(Skeleton skeleton)
{
// right hand in front of right shoulder
if (skeleton.Joints[JointType.HandRight].Position.Z < skeleton.Joints[JointType.ElbowRight].Position.Z && skeleton.Joints[JointType.HandLeft].Position.Y < skeleton.Joints[JointType.ShoulderCenter].Position.Y)
{
// right hand below shoulder height but above hip height
if (skeleton.Joints[JointType.HandRight].Position.Y < skeleton.Joints[JointType.Head].Position.Y && skeleton.Joints[JointType.HandRight].Position.Y > skeleton.Joints[JointType.HipCenter].Position.Y)
{
// right hand right of right shoulder
if (skeleton.Joints[JointType.HandRight].Position.X > skeleton.Joints[JointType.ShoulderRight].Position.X)
{
return GesturePartResult.Succeed;
}
return GesturePartResult.Pausing;
}
return GesturePartResult.Fail;
}
return GesturePartResult.Fail;
}
}
Notice it goes through a series of joint checks and returns one of three results:
Succeed: The gesture has completed in full (fired when all the 'if' statements succeed)
Pausing: The gesture is partially complete (fired when you want to indicate the gesture is "in motion" but not complete)
Fail: The gesture is not being executed at all
When you have the gesture you want initialized you just need to send the 'skeleton' data to the gesture library:
private void OnSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
using (SkeletonFrame frame = e.OpenSkeletonFrame())
{
if (frame == null)
return;
// resize the skeletons array if needed
if (skeletons.Length != frame.SkeletonArrayLength)
skeletons = new Skeleton[frame.SkeletonArrayLength];
// get the skeleton data
frame.CopySkeletonDataTo(skeletons);
foreach (var skeleton in skeletons)
{
// skip the skeleton if it is not being tracked
if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
continue;
// update the gesture controller
gestureController.UpdateAllGestures(skeleton);
}
}
}
Finally, execute on the gesture:
private void OnGestureRecognized(object sender, GestureEventArgs e)
{
switch (e.GestureName)
{
case "SwipeLeft":
// do what you want to do
break;
default:
break;
}
}

Categories