Kinect paint - hand tracking c#, kinect SDK 1.7 - c#

I am trying to make some simple app which allows you to paint on canvas with your right hand.
Fortunately I know how to make painting function but I have a little problem with other thing.
As you know SDK provides you to use a control named KinectRegion which has KinectCursor which is the representation of user's hand.
The problem is that I don't know why when I am trying to paint something my painting path starts in different position than my KinectCursor is ?
I don't have this problem when I use my own right hand mapping function but in that case I can't use other things like KinectCircleButton because I don't have KinectRegion.
Anyone know how to get or to map KinectCursor position(x,y) from KinectRegion ?
visualisation of my problem:
[IMG]http://i58.tinypic.com/iqgemt.png[/IMG]

I'm working on the similar project on painting with Kinect. Actually the position you need is in the HandPointer. You can get the position of your hand relative to UIElement by a method called GetPosition(UIElement element) which obviously takes that element as parameter.
An example of using the method looks like this:
public partial class MainWindow
{
public Point position;
public MainWindow
{
KinectRegion.AddHandPointerMoveHandler(this, OnHandPointerMove);
}
private void OnHandPointerMove(object sender, HandPointerEventArgs e)
{
position = e.HandPointer.GetPosition(myCanvas);
}
}

Now the thing is your hand position and kinect hand position has a gap in both X and Y. Try to make these gaps zero. Then your hand will exactly map the painting point. For example output x,y coordinates at once for both hand point and painting brush point. and get the individual gap for both x coordinates and y coordinates. now these difference should be deducted form x and y respectively. So then your hand and paint brush points will map accordingly.

Related

Projecting points onto 3D terrain surface in UWP MapControl

I'm creating a windows desktop app using the UWP Map control (MapControl) and inking based on this blog post.
I have the basics working. can draw ink on the canvas and convert them to map locations, but I can't get the points to correctly project onto the map 3D surface.
I'm using mapControl.GetLocationFromOffset to transform the control-space offset to map position but no matter what AltitudeReferenceSystem I use the points are offset above or below the 3D mesh. I'm basically trying to emulate how inking works in the built in windows Maps app. But since there's a pause after drawing each stroke I'm thinking it's doing some ray casting onto the 3D mesh. Lat/Long are correct but altitude has an offset.
Geopoint GeopointFromPoint(Point point)
{
Geopoint geoPoint = null;
this.map.GetLocationFromOffset(point, AltitudeReferenceSystem.Geoid, out geoPoint); // Tried all alt ref systems here
return (geoPoint);
}
...
BasicGeoposition curP;
foreach (var segment in firstStroke.GetRenderingSegments())
{
curP = GeopointFromPoint(segment.Position).Position;
geoPoints.Add(curP);
}
...
This is my inking around a patch of grass in the Maps App which determines correct altitude:
And the same area using the Map control + GetLocationFromOffset, blue ink is annotation showing altitude offset:
How can I project screen/control space coordinates onto the 3D mesh in the UWP Map control and get the correct altitude?
Edit: And the answer is I'm an idiot, have been thinking in meters from center of earth too long, ie I thought altitude was the altitude above sea level of the map location, but it's not, it's the points altitude above the map. So just setting altitude to zero works.
Just setting the altitude to zero may still produce unexpected results. The call to GetLocationFromOffset is where your issue is. (you should really use TryGetLocationFromOffset BTW and handle failure cases - it isn't always possible to map a screen offset to a location, for example when you click above the horizon).
The AltitudeReference you pass is is telling the control what to intersect with. You can think of it as a ray you're shooting into the 3D scene at that screen pixel. The ray will intersect the Surface first, then the terrain, then the Geoid or Ellipsoid depending on where you are. You were asking for the intersection with the Geoid, which probably wasn't what you really wanted. You probably want the intersection with the surface for an inking case. The docs aren't terribly clear on what this parameter does and should probably be updated.
If you reset all altitudes to 0 with an altitude reference of surface for the line you pass in, it will always put the line on the surface, but that might not match your ink.
There may actually be a bug here - if your code snippet above was passing the Geopoint returned from GetLocationFromOffset to the polyline, it should appear at the same pixel (you should be able to round-trip values). What was the AltitudeReferenceSystem and value returned for these GeoPoints?
First problem was I wasn't passing a AltitudeReferenceSystem to the Geopath constructor while translating ink strokes to map polygons. Once you do this you can pass the result of TryGetLocationFromOffset directly to the polygons.
Second is a possible bug with the map control. If the map control does not take up the entire height of the main window you see a pixel offset between what TryGetLocationFromOffset and where the ink stroke was drawn.

How can I draw a Line animation to outline various shapes?

I am trying to draw a line animation outline various shapes as in the image below. I am very well aware that it's best practice that I mention what I've been able to achieve to get specific help, but I am not sure where to begin, just that I know that using a Line Renderer could be a good approach to achieving this. That said, how can I achieve this?
UPDATE
I think I didn't explain a few things clearly enough. I am interested in animating the outline of objects without arrows, just a line traced round the outline like the image below:
I would do the following: (pseudocode, untested)
For every prefab or gameobject, store a List of edges that define your outline.
I wouldn't recommend using the mesh's edges, it's probably better to have a specific predefined list of edges per shape to avoid the inner edges of the object. Every entry in the list is defined by two Vector3's which are the two vertices.
List<Vector3[]> outline = new List<Vector3[]>();
Now, you have many ways to actually draw the arrows, like having them as individual gameobjects (probably not a good idea), particle system, or just drawn automatically from the parent objects update function. I would recommend the latter.
Now you would store a bunch of floats that define where your arrows are
public List<float> arrow_locations = new List<float>();
//adding one arrow
arrow_locations.Add(0.0);
//now in the update function of your parent object, update the arrow locations
private float cycle = 0.0f;
void Update()
{
float segment_size = 360.0f/outline.Count;
for(int i=0; i < arrow_locations.Count; i++)
{
arrow_locations[i] += 0.05f; //speed of spinning
if( arrow_locations[i] >= 360.0f ) arrow_locations[i] = 0;
//now to get the actual location of the arrow
int which_edge = Mathf.Floor((arrow_locations[i]/360.0f)*outline.Count);
//this will give us a number 0..1 telling us where along the edge the arrow is
float weight_within_edge=(arrow_locations[i] - segment_size*which_edge)/segment_size;
//here we lerp between the two vertices of the edge
Vector3 new_loc = outline[which_edge][0]*(1.0-weight_within_edge) + outline[which_edge][1]*(weight_within_edge);
//now that we have the location of the arrow, draw it
//note, you can get more efficient if using instancing for all arrows
//You can also use line drawing, but i wouldn't recommend that.
DrawMesh(arrow_mesh, new_loc, Quaternion.identity);
}
}
Please note, that when you have the positions of the arrows, you can opt to draw them in 2D in the UI by projecting them onto the camera plane. The lines aside from the arrows are themselves static, so you can draw them as part of the mesh very easily. Also note, I make no mention of the objects position, all values should probably be defined in local space, then transformed with the object. You can transform the drawn stuff in the the DrawMesh function by supplying a a transform matrix.
I think a shader with a parameterized radial mask would be the best way to do this. I have never done one myself, so I only have a general idea of how it's done, but here is how it would work AFAIK:
Create some kind of cell shader that can draw the edges of objects.
Create a filter/mask that has an angle shape extruding radially from the center to the edges; you can control the shape/angle using a parameter. Unity already has something similar to this in the Tanks! tutorial - Tank Health lesson.
Note: The tutorial might even be exactly this idea, but I don't remember with enough details to confirm; I'll update the answer after I take a look again.
The tutorial has the same idea, but it applies it using unity's builtin UI stuff.
Using this mask, only the masked area of the shape's edge will be drawn the screen.
By increasing the angle parameter of the mask over time, you can create the effect of the edge of the object getting revealed radially over time. Which seems to be exactly what you want.
To help visualize, a very professional diagram made in paint:
light blue = mask.
dark blue = "revealed" part of the mask (angle parameter). Plus how it would behave if the angle is increased (arrow).
green = object.
black = outline being drawn to the screen.

How to animate a simple 3D shape using Helix Toolkit?

I'm trying to animate a simple rectangular shape so that it scales in size in a certain direction. As it is, I am making a rectangle that extends from point A to B. The end goal is to animate it so that it starts at A and is transformed to be the length required to get to B.
I'm pretty new to animation in general, so this process seems finicky to me.
Right now I am:
Creating a vector between the start and end point
Finding the 8 corners of the rectangle along that vector
Creating 2 triangles for each face of the rectangle
Rendering the shape
This is all being done by using a MeshBuilder object and adding the triangles and points individually.
So, the way I'm creating the prism doesn't really help for what I need to do. Ideally I suppose, I would just create a short prism aligned between the points, and then just extend the rectangle to be the right length in an animation.
Any thoughts?
I solved this is a sense by scaling the 3D object from a size of 0 in the X/Y/Z to 1.0. So instead of the prism "extending" from A to B, it more or less "grows" to B.
Note that the ScaleTransform3D needed to have the CenterXYZ properties set to the coordinates of point A in order for it to be anchored to the correct position.
If I find a better solution, I'll update this answer later.

2d Sprite Animations without using XNA or other third-party libraries

I want to create a simple game, similar to what can be created with RPG Maker. What I am primarily looking for at the moment is a tutorial which can guide me on how to accomplish it without using XNA or any other specific library.
The ideal would be a step-by-step tutorial or a good, easy-to-understand example.
My goal is to do it on my own, and come to an understanding of how the overall process works with regards to animation elements relating to one another.
I've already built a simple snake game in the past using a simple tutorial, but there was no movement animation in it and that's the primary thing I'm wanting to learn next.
Edit:
All of the tutorials I have found so far use third-party libraries. I did come across this link, but I'm looking for something which doesn't include XNA.
There are a number of ways to approach the topic you're describing. I'm going to give a bit of an overview, and then hopefully provide some resources which can give you examples to get you started.
Essentially, sprite-based animations revolve around having a series of similar images which, when displayed sequentially, create the appearance of motion, similar to a flip-book.
The trick is to understand when your sprite is moving (and therefore should be animated) - and when it is standing still (and therefore should not be animated). In other words - assuming that your game's character is only supposed to move while you hold ▲, ▶, ▼ or ◀, you need to detect when one of those keys starts and stops being pressed, so that you can start/stop your animation accordingly.
Imagine that for simplicity, you only have 2 sprites. The first (left, below) represents your character standing still, and the second represents your character mid-step (right, below):
When the ▶ button is not pressed, you simply continually display the first image. When the ▶ button is pressed, you toggle between the two every x milliseconds (depending on how fast you want the animation to appear).
An animated .gif is one format in which you can contain a series of simple image frames, which are intended to be displayed as a series (and therefore create the illusion of animation). If you were to create your sprites using this format, you could use code similar to that found in this SO discussion, which provides some example code for how to use C# to animate an animated .gif and control its start/stop.
Alternatively, if you wanted to use a sprite file (like the one I included above), you could use something similar to this CodeProject code, which focuses on GDI interaction with the Windows environment in order to extract and paint a portion of the sprite onto a target canvas. By repainting every x milliseconds (as mentioned above), this can provide the same effect.
A few things to keep in mind:
You'll need to handle transparency in your sprites (the Mario sprite above, as an example, has a transparent background) - so that the background of your game environment shows through. If using GDI - this all has to do with how you call the painting methods. If using an animated .gif - the method to use depends on how you display it on your window.
For some additional resources / examples / references, check out the following resources:
Intermediate C# Game Making Tutorial - 2 - Sprites
2D Game Primer (Visual C) - an older article which talks quite a bit about the concepts of sprite animation, timing and such - and gives some example code (some DirectX in examples)
Sprite.cs - an example of some C# manipulation code for dealing with sprites and sizing (uses OpenGL, so may not be applicable)
And for Sprite development:
SpritePad - a tool for creating Sprite Sheets
http://makeagif.com/ - a tool for creating animated gifs online
http://picasion.com/ - another animated gif creator
I threw together than example of what I think it is that you were after. This example can be applied to buttons or picture boxes. I chose this way of out of simplicity.
Each instance of an animation holds a timer, and a list of images. Updating the image of the target control whenever the timer fires its event.
I have uploaded my project file here. http://mcspazzy.com/code/ParTest.zip
Hopefully it is enough to help. Just ask if you need more explanation.
The class
public class Animation
{
readonly Timer _animtimer = new Timer();
public List<Image> Frames;
public int FrameIndex;
private Button _target;
private PictureBox _ptarget;
public void Target(PictureBox target)
{
_ptarget = target;
}
public void Target(Button target)
{
_target = target;
}
public int FrameSpeed
{
get { return _animtimer.Interval; }
set { _animtimer.Interval = value; }
}
public Animation()
{
Frames = new List<Image>();
_animtimer.Interval = 100;
_animtimer.Tick += Update;
}
public void Play()
{
_animtimer.Start();
}
public void AddFrame(string file)
{
Frames.Add(Image.FromFile(file));
}
public void Stop()
{
_animtimer.Stop();
}
private void Update(object sender, EventArgs eventArgs)
{
FrameIndex++;
if (FrameIndex == Frames.Count)
{
FrameIndex = 0;
}
_target.Image = Frames[FrameIndex];
_ptarget.Image = Frames[FrameIndex];
}
public static implicit operator Image(Animation a)
{
return a.Frames[a.FrameIndex];
}
}
This was in my Form load. Can really go anywhere that stuff is initialized.
private void Form1Load(object sender, EventArgs e)
{
_testAnim.AddFrame(#"F:\Im\Black.png");
_testAnim.AddFrame(#"F:\Im\Blue.png");
_testAnim.AddFrame(#"F:\Im\Green.png");
_testAnim.AddFrame(#"F:\Im\Orange.png");
_testAnim.AddFrame(#"F:\Im\Red.png");
_testAnim.Target(ButtonTest);
_testAnim.Target(PicBox);
_testAnim.Play();
}
On quick search I found this example.
I'm not very experienced in c# graphics, but here are few points I have learned working with non-graphic oriented languages:
Declare where/what you want to draw
Create a loop to run until abort/event to end the loop (like object colliding with something)
In the loop: wait, clear the old drawing area, recalculate new position, draw in new position
If needed you can change the image to draw too, but then you need separate images for each "frame" you want to have
multithreading is a good idea, so you can separate running the graphics from other game logic
try keeping the time from clearing the drawing area and re-drawing as short as possible to prevent flickering
Keep track of the size of objects you draw, makes it easier to check for collision (like center of the sprite + radius, then you can easily calculate a circle area around it to check if two sprites are too close to each other)

resolution independent grid for animations?

I'm working on a game made in XNA,C# and I want to enable xml based animations.
XML will look like this
<Animation>
<AnimatedObject>
<Filename>Spaceship_Jet_01</Filename>
<Flipped>false</Flipped>
<StartPosition_X>300</StartPosition_X>
<StartPosition_Y>500</StartPosition_Y>
<GOTOPosition_X>650</GOTOPosition_X>
<GOTOPosition_Y>500</GOTOPosition_Y>
<Time>10000</Time>
</AnimatedObject>
</Animation>
This will move an object to the side, like this
http://imm.io/odc7 (sorry the X coordinate is wrong)
I noticed there will be problems, when the players display resolution is different from mine because I enter pixel precise information about where the object comes from and where it has to go.
I thought about a grid so I can tell the programm to move the object from (30,27) to (22,27) e.g.. Is this a good solution? The grid has to be independent from the resolution but the number of tiles has to be constant and I have to draw the object to the screen. That means I have to find the right pixle position of the tile at position (22,27) and then "move" the object to that tile.
Is there a better way to do that? How can I solve this with XNA?
If you use a 2D camera you won't have any problem... because calculating the new view to adapt it to the new resolution is not difficult.... and you have not to change anything of your loads methods nor logic...
You can do, but I don`t like
Work with positions in [0..1] range, is difficult to measure.
Fix the position with the new resolution factor when you load the xml... is ugly...
Pos *= NewResolutionSize/DefaultResolutionSize;

Categories