I am trying to make a app in c# that will detect / classify 3 poses of human body which are standing, sitting and lying. I can correctly detect / classify 2 of them (sitting and standing) with skeleton tracking. When it comes to lying on the floor, Kinect seems to not be able to track skeleton of a human body.
Does anyone have any experiences with skeleton tracking in lying position? As soon as I lye down, I am loosing joints positions. Is this task impossible? Thank you.
The Windows SDK for Kinect V1 is not great at recognizing people lying down.
V2 improves a lot on this, I would recommend you to participate in the beta or wait for V2.
One possible solution with V1, is to place a second camera vertically on the floor, use seated mode - so it detects on motion - and see if you find a person with that camera and not on the other, then it is likely lying down.
I have not tested this solution, but in theory it can work - try to set it up yourself and see if it fits your scenario needs.
Related
Ok here is what Im trying to achieve - when user's feet collide with grass mesh, the mesh is secured to the ground at the base but the tops move away, then bounce back when foot is no longer there:
This example, and everything I've found however, are using billboarding/2d as grass. I want 3d, like these:
https://assetstore.unity.com/packages/3d/vegetation/plants/grass-toon-76674
Where some sort of collision is occurring, or Ive tried experimenting with unity's trees. Trying to find the most CPU/GPU effective (for Oculus app or mobile) way to do this.
Is this possible - to have interactive, 3d not 2d grass?
The only thing I can think of is to model each blade of grass as several bars connected by universal joints (I'm not sure what kind they would be in unity, but that is the mechanisms term).
Each joint could have a torque proportional to its angle, so the more you bend it, the harder it tries to flick back. Then its just a matter of adding forces to the blade structures as you bump into them.
This is more of a engineering solution than a programming one, and would require a little bit of math, so its probably not ideal. but it should work.
I am trying to create a simple scene where a few objects are placed on the table. Object placement works perfect but when I move the device, the objects drift around a bit. Which at one point makes the objects placed at the corner feel like they are not on the table but floating in the air.
Even in the sun moon and earth example in Unity examples here: https://github.com/googlesamples/tango-examples-unity
The earth n moon drifts as you move the device
Is this a bug or is there any special setting which I'm missing?
The objects drift because as the Tango device moves through space, it is only tracking its own position in 3D space. For objects to remain static in a dynamic environment, the device needs to understand the position of the placed objects in 3D space and their relation to the surroundings in order to anchor the objects and reduce drift.
Luckily, TangoCore has you covered here and the 3 Core technologies of Motion Tracking, Depth Perception and Area Learning all work together to help out.
If I'm not mistaken, the Sun and Moon example is the scene "SimpleAugmentedReality" under tango-examples-unity / UnityExamples / Assets / TangoSDK / Examples / Scenes /
However if you would like to anchor the objects in 3D space and reduce drift, you'll need to use Area Learning and Depth Perception as well. Area learning performs Loop Closures as the device realises it has "seen" an area before and adjusts the path and markers to provide a more accurate device and augmented content position.
So here is what you can do to learn what you need to. Save your current scene, go to open Scene and follow this path tango-examples-unity / UnityExamples / Assets / TangoSDK / Examples / Scenes / and load up some of the other scenes to get an understanding of how the technologies intertwine.
For example, you could load up the ExperimentalMeshBuilderWithColour scene, and learn how the Depth Processing works programmatically, and then load the MotionTracking scene and learn how to access and use Motion Tracking from the TangoManager Game Object. And finally (also probably most frustratingly difficult) learn how Area Learning is managed with the AreaDescriptionManagement and AreaLearning scenes.
This will not only solve your drift issues, but also give you a much fuller understanding of the capabilities of the Tango Technology and allow you to express your ideas much easier.
I'm now detecting all the skeleton in a wpf application, I want to know how to detect the fingers to appear with the skeleton? I'm using microsoft Kinect for windows sdk ver 1.5
Many thanks
The Kinect unfortunately is not sensitive enough to recognize fingers so the library will not provide that as part of the skeleton. Maybe the Kinect 2.0 rumored to come out with the Xbox 720 will be able to provide that level of detail.
Candescent NUI might be what you're looking for. As OpenUserX03 said, however, the Kinect isn't ideal for this task. Perhabs you should have a look at the coming-up LEAP technology, which specializes in finger detection.
The cameras on the Kinect are not meant to be able to do joint tracking for the hands to that level of detail. Tracking the individual fingers is possible but wont be very reliable. To represent a players hand in the skeleton, you can check if the players hand is opened or closed. A possible way to see if the hand is open or closed would be to do pixel checks in an area surrounding the hand. This way with some tuning you could calculate how much of that area is the hand (using the depth and color stream) and how much is not. For example: If 40% of that area is the same depth as the hand joint, the hand is closed in a fist. If 70% of that area is the same depth as the hand joint, the hand is open. Then you could possibly use the angle of the elbow and wrist joint to be able to represent a closed or open hand at that angle on the skeleton.
I want to develop a "People Counting System" using OpenCV (or Emgu CV).
Please guide me on how to implement or lead me to some examples or open source projects.
(I have done some work: extracting diff then threshold to delete background, using motion history and like that; still no good results.)
Edit 1: I am counting a high people flow (a dozen of them may come through simultaneously).
Edit 2: It must be at least 80% accurate. People are walking through a door that is almost 5 meters wide. The problem is I have no control on the position or angle of the camera. Camera is shouting the place from a 10m distance at a 2.5m height.
Thank you
If you call a people counting system a system that counts people that are in a room then I recommend you implement the hardware with a microcontroller with 2 lazers(normal lazer toys work) and 2 photoresistors.For the microcontroller I recomen you use Arduino.And then make an C# application that has a SerialPort object and reads the data that the arduino sends through the USB.The arduino will send 1 for "someone entered the room" and 0 for "someone left the room" for example.Then the logging and statistics can be done easily in C#.
Arduiono Site:here
Photoresistor for $1: here
This solution is alot cheaper and easyer to implement than using a camera that is with a fairly good quality.
Hope I helped you.
Check out the HOG pedestrian detector that comes with recent versions of OpenCV (>= 2.2).
See modules/objdetect/src/hog.cpp and samples/cpp/peopledetect.cpp in the OpenCV sources. Unfortunately there is no official documentation about it yet.
This would help you to count moving things including people: Motion Detection project on CodeProject
Are people the only kind of "entities" in the scene? If this is not the case, do you care about considering a person some other kind of thing that moves through the scene? Because if that is the case, you could just count blobs that come in or come out from the scene. It may sound a bit naive but I will take some kind of motion image, group motion pixels by distance in clusters. Your distance metric could take into account some restrictions, such as that people will "often" stand so pixels in a cluster should group around some kind of regression line (an straight-up line if the camera is aligned with de floor). It shouldn't be necessary to track them in the scene, just noticing when they enter or they leave, though you'd get some issues with, for example, people entering on their own in the scene and leaving in pairs or in groups... Good luck :)
I think if you have dense people crowd with a lot of occlusions you have to use some machine learning algorithm, for example you can use Implicit Shape Model for features.
It really depends on the position of the camera. Assuming that you can get front facing profiles of the people in the images:
This problem is basically face detection and recognition.
There are many ways to go about finding faces, but this is the approach that I'm a little more familiar with.
For the face detection you need to do image segmentation on the skin tone color. This will extract skin regions. [Arms, the chest (for those wearing V cut tops), face, legs, etc] Then you would need to line up the profiles of the skin regions to the profile of your trained faces.
[You'll need to use Eigenfaces to create a generic profile of what a face looks like]
If the skin region lines up and doesn't devate too far from the profile, then it is considered a face. Once the face is confirmed, then add it into the eigenfaces data store [for recognition]. To save processing you might want to consider limiting the search area if you are looking for a previous face. [Given the frame rate, and last time the person was seen]
If you are referring to "Crowd flow" I think you just mean the density of faces in a crowd.
Now you've confirmed that a moving object in the video is a person. Now you just need to note that and then make sure that you don't consider them as a new person again.
This approach: Really depends on your ability to detect face regions. This may not work if the people in the video are looking down, not fitting the profile of the trained data etc. Also it may be effected if a person puts on sunglasses within the video. [Probably would be considered a "new face"]
I'm currently trying to implement a marble maze game for a WM 5.0 device and have been struggling with developing a working prototype. The prototype would need the user to control the ball using the directional keys and display realistic acceleration and friction.
I was wondering if anyone has experience with this and can give me some advice or point me in the right direction of what is essential and the best way to go around doing such a thing.
Thanks in advance.
Frank.
When reading your answer I didn't get the feeling you are looking for a game framework, but more: how can I easily model a ball with acceleration and friction.
For this you don't need a full fledged physics framework since it is relatively simple to do:
First create a timer which fires 30 times a second, and in the timer callback do the following:
Draw the maze background
Draw a ball at ballX, ballY (both floating point variables)
Add ballSpdX to ballX and add ballSpdY to ballY (the speed)
Now check the keys...
if the directional key is left, then subtract a small amount of ballSpdX
if the directional key is topleft, then subtract a small amount of ballSpdX and ballSpdY
etc
For collision do the following:
first move the ball in the horizontal direction. Then check the collisions with the walls. If a collision has been detected, then move the ball back to its previous positions and reverse the speed: ballSpdX = -ballSpdX
move the ball in the vertical direction. Then check the collisions with the walls. If a collision has been detected, then move the ball back to its previous positions and reverse the speed: ballSpdY = -ballSpdY
by handling the vertical and horizontal movement separately, the collision is much easier since you know which side the ball needs to bounce to.
last nu not least friction, friction is just doing this every frame: ballSpdX *= friction;
Where friction is something like 0.99. This makes sure the speed of the ball get's smaller every frame due to friction;
Hope this helped
I would recommend checking out XNA Studio 3, it has built in support for PC, Xbox 360 and mobile devices, and it's an official & free spin-off of Visual Studio from Microsoft.
http://creators.xna.com/en-US/
http://blogs.msdn.com/xna/
If you search around, people have written tutorials using physics (velocity on this one)
http://www.xnamachine.com/2007/12/fun-with-very-basic-physics.html
Try XFlib. It is in c++, but most cool things for the mobile have to be in c++, unfortunately. The site has some very cool free games. You can also see the source of most of the game too. Many have the physics you want.
Unfortunately, XNA doesn't support the windows mobile platform. However, as it seems that you're not having a problem with the technical issue of drawing on the WM device, but with the logic required to implement physics based movement, then it's not a bad idea to consider XNA to prototype the physics and movement code.
Check out some of the educational topics at creators.xna.com, and also "gamedev.net"
If you are at a loss, there's no mistake in trying a "lighter" tool for prototype. I would try Torque Game Builder - it spits out XNA, although maybe not meant for your platform.
At the Samples of the Windows Mobile SDK (check out the WM 6.0 SDK too), there are a couple of game applications. One of them is a simple puzzle game; not much, but it is a starting point.
The use of physics in game development is not specific for Windows Mobile. You can find a huge literature about this subject. This comes up in my mind now. If you are serious about game development, in any platform, you should do a little research first.
I dont know if this may help but i saw a Marble application for the Android platform on google code. Check it out here, it may throw some insight on the actual logic of the game.
The code is open sourced and written in java (using the android sdk) put nevertheless it may be useful. Also to better understand the code checkout the documentation for the SensorsManager, SensorEvent etc here
I wouldn't recommend using the same architecture as this application thou.