Ask about kinect fusion for AR - c#

I want to create an AR application using Kinect Fusion.
I want to insert a 3d model into my kinect fusion reconstruction. The expected output is like these two videos:
https://www.youtube.com/watch?v=gChlRebNloA
https://www.youtube.com/watch?v=MvP4cHfUD5g
How to overlay the 3D object onto the 3d reconstruction?
Is there function in the SDK that can be used to achieve my goal?
Thanks
ps: I use C#

You should probably start to explore the KinectFusionExplorer-WPF KinectFusion available on codeplex:
http://kinectforwindows.codeplex.com/SourceControl/latest#v1.x/ToolkitSamples1.8.0
KinectFusionExplorer-WPF has some nice utility methods for matrix transformations, (virtual) camera configuration, etc.
The process for doing the AR augmentation is:
viewport: set your viewport using the same resolution as the one for the camera image,
video background: retrieve the video image from the Kinect and displays it in the background (either as done in the SDK examples or using a textured quad with the video image as done more traditionally),
geometric registration: use WPF3D and set a Camera object using: the intrinsic parameters of the Kinect camera (or the same ones used by KF), the camera pose you get via GetCurrentWorldToVolumeTransform() from your volume,
rendering: set any local transformation you want for your model (xaml) and render (using the way it's generally done with WPF3D).

Related

AR Multiplayer: How to share the position of the 3D object, relative to the image target?

I'm trying to implement a very basic ARCore-ARKit multiplayer in Unity. I can use markers to place 3D content over them using Vuforia or ARKit/ARCore image recognition.
But how can I store and share across several devices (iOS and Android) the position of the 3D object, relative to the image target?
I can't find any useful tutorial on this topic.
You can make the 3D objects child objects of your image anchor and share the local position. For the sharing you can use Unity networking

Animating a sprite

Im developing a game like "bubble-bobble". So far I have done physics and collision detection.
Now I want to make my Hero (Rectangle Sprite) animated. I would be glad if someone could explain simple scripting for simple animated characters or some nice links for animation.
The XNA Documentation includes an entire article on Animating a Sprite. The basic technique is to use an AnimatedTexture class, which is included within the Animated sprite sample code.
The high level idea is that you load a texture into memory using a graphics API. Since you're using C#, this is most likely done through XNA.
This texture you have loaded contains each frame of animation that is required, and may span across multiple textures. When you go and render your 'sprite' object, you pass the XNA API the texture you want to use, and a source rectangle coordinates that surround the specific frame of animation you want within that texture.
It's up to you to manage this process. I create tools that assemble these source rectangles and stores meta data about each specific animation each sprite has; like which rectangles, and the duration of each frame, etc.

Is there any methods to get 3D image from webcam in c#?

I want to calculate depth of an image. So that i can eliminate far objects from the image.
Is there any methods to do so in c# with single camera??
This website shows how to get a webcam image using C#. However, just like a photo, it is flat so there is no way to distinguish objects at different distances from the camera. In general, with just one camera and a single photo/image, what you want is impossible.
With one or two cameras that snap two images/photos with some distance in between, you can distinguish depth (just like you do using your two eyes). However, this requires very complex mathematics to first identify the objects and second determine their approximate distance from the camera.
Kinect uses an infrared camera that creates a low-resolution image to measure the distance to objects in front of the camera, so that it can distinguish the player from the background. I read somewhere that Kinect cameras can be attached to a normal computer, but I don't know about the software or mathematics you'll need.
If you illuminate a straight line with a laser at an angle to the scene, the displacement of the line will correspond exactly to the height of the object. This only gives the height along a single line, subject to the resolution of your camera. If you need a complete 3D scan you'll need to move the laser and take multiple pictures.
a c# reference would be needed for each frame as the streaming video file comes in. at the start of the streaming the subject will need to turn their head and spin so a series of measurements can be captured from the subject. this then could be feed to a second camera like unity 3d a virtual camera that transposes a 3d image over the top of the streamed image. there are a lot of mobile phone apps that can capture 3d objects with a series of still frames i had one on my galaxy s6 also the galaxy s6 and up has a depth chip in their cameras that they sold to i phone this is used on the apple 3d camera. i have been thinking about how to do this also would love to email you about it. to note it would be a similar concept to facial recognition software.

Interacting with avatar using Kinect and Unity

I want to move the avatar based on the movement the player using kinect and Unity, are there any good tutorials?
We are using unity and Kinect interface to create a simple application. Based on the movement of the player we need to move the avatar.
We are supposed to use Unity with GAKUNITY, No OpenNI or any third party tools.
Are there any good tutorials for GakUnity with Kinect?
GAK means Gadget Accelerator Kit
We just want to move any avatar with player movement in front of kinect interface. Even help in hand movement also highly appreciated.
You can also share useful links or books regarding to unity and kinect programming.
Custom Kinect Gesture Recognition using OpenNI and Unity3Dkinect.dashhacks.com
A tipster has tipped a lil tipple my way in the form of custom gesture recognition for the Kinect using OpenNI and Unity 3D. What this allows you to do is create you own custom gesture that are recognized by the kinect through the software interface.
Unity and Kinect tutorialnightmarekitty.com
For this chapter, we are going to be using a very popular game engine called Unity. By integrating OpenNI, Nite and Sensor Kinect into Unity we will control a 3d character and multiple user interfaces. After we cover the main Components, we will build an example of each from the bottom up.
Kinect Wrapper Example Projectwiki.etc.cmu.edu
This scene shows you how a skeleton is generated / tracked by placing spheres at each of the bones tracked by the Kinect, and how to use kinect to control your model. Use this to get a feel for what the Kinect is capable of. It also shows you how to prepare your GameObjects
I am not familiar with Unity 3d,but you can try using kinect with XNA. At the latest version of Kinect for windows SDK 1.5 and Developer Tookit 1.5.1,there is a sample demonstrating how to interact with 3d avatar using Kinect and XNA,you can find more information on http://msdn.microsoft.com/en-us/library/jj131041
I have no idea about GAKUNity, but you can use zigfu plugin for unity3d and do your project. It has got sample codes and character which you can use for reference, and the sample codes are commented nicely .
You can just import your bone, drag your bones on the zigskeleton script . just read through th zig skeleton script and you will understand the function and structures they are using.
And you can even refer the MEET THE KINECT book, they have explained about using zigfu in unity .

face detection using a webcam

i am developing a 3d project and would like to include the following feature :
As my webcam is watching my face, if i move to the left or the right the projects camera position moves to the left or the right to create an "look-around-the-corner" effect .
Does anyone know a face detection project in .NET c# ?
You can use OpenCV for .NET - there is a wrapper for .NET which also comes with a sample application doing face recognition for images - easy to adapt for the camera if you can extract the samples.
Are you familiar with web services? Depending on how often you need to scan for faces during your webcam stream, you could grab a frame, send it to a face detection web service and it would return the coordinates of faces in that frame.
You could use http://detection.myvhost.de/ because its free!
bafta ;)

Categories