Kinect Custom Cursor - c#

I need to make a cursor in Kinect, but I don't know where to start. I need to make it using WPF and C#. Cursor should be in shape of hand and when I hover over element the "loading" circle should appear, and when it "loads" it should fire click event on hovered element.
I'm sure that your're all familiar with this.
It would be of great help if someone could write me some directions oh how to accomplish this.

Here is a code snippet that might help you:
using Microsoft.Research.Kinect.Nui;
Runtime nui = Runtime.Kinects[0];
nui.Initialize(RuntimeOptions.UseSkeletalTracking);
nui.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);
void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
SkeletonFrame sf = e.SkeletonFrame;
SkeletonData d = (from s in sf.Skeletons
where s.TrackingState == SkeletonTrackingState.Tracked
select s).FirstOrDefault();
if (d != null)
{
SetHandPosition(imageCursor, d.Joints[JointID.HandLeft]);
}
}
void SetHandPosition(FrameworkElement e, Joint joint)
{
Joint scaledJoint = Coding4Fun.Kinect.Wpf.SkeletalExtensions.ScaleTo(joint, 600, 400, 0.75f, 0.75f);
Canvas.SetLeft(e, scaledJoint.Position.X);
Canvas.SetTop(e, scaledJoint.Position.Y);
}
If you want your cursor to be different hovering on an element then just go to the elements properties and set a cursor for that element. In Visual Studio you can choose a cursor in the elements properties.
To make a click on a hover over an element you have to implement the MouseEnter event
private void button1_MouseEnter(object sender, MouseEventArgs e)
{
....
}
Here are some infos about it:
button1.PerformClick() in wpf

Here is a nice solution using the official SDK, but even if you're not, it can be very (!) helpful (it was for me):
You should take a look at the (free) code available here Beginning Kinect Programming with the Microsoft Kinect SDK sample code you click on "Source Code/Downloads" and what's is going to interest you in the sample is the Chapter 6 (name of the folder).
Basically they're using a static class KinectInput that allow to raise new event like KinectCursorEnterEvent, there is a cursor manager KinectCursorManager which does almost everything get the hand position/update the cursor ... and They use an adorner to put the cursor, with the FrameworkElement you want as a cursor.
They implement the HoverButton you're talking about. It fires the click event after a timer elapsed (timer launched when the KinectCursorEnterEvent occured). The solution they propose is elegant, and allow an easy implementation of nice controls.
You can easily modify the code to handle the two cursor (that's the value-added of the Kinect, isn't it?)
having an Enumeration CursorSide:
public enum CursorSide
{
Left,
Right
}
and modifyong only the KinectCursorManager having a Dictionary of capacity 2, and the enumeration being the Key, and having a pointer on the elemtn under the cursor for each hand:
private Dictionary<CursorSide, CursorAdorner> _cursorAdorner;
private UIElement _lastElementOverRightHand;
private UIElement _lastElementOverLeftHand;
But before you have to remove the part of the code that does the selection of the primaryHand (basically the hand closest to the Kinect).
I hope this can help somebody :-]
The book is very interesting, you can buy it for a few bucks.

Related

Surface sdk 2.0 keyboard not showing in application

I am trying to show the keyboard when I click a button, but it's not showing a keyboard at all.
The "TEST" gets printed but the keyboard isn't showing.
My code is :
private SurfaceTextBox mySurfaceTextBox = new SurfaceTextBox();
void showKeyBoard(object sender, RoutedEventArgs e)
{
//System.Windows.Input.Keyboard.Focus((IInputElement)getCanvasFromButton((SurfaceButton) sender));
System.Windows.Input.Keyboard.Focus((IInputElement)mySurfaceTextBox);
Console.Write("TEST");
SurfaceKeyboard.IsVisible = true;
SurfaceKeyboard.CenterX = (float)InteractiveSurface.PrimarySurfaceDevice.Bounds.Width - (SurfaceKeyboard.Width / 2);
SurfaceKeyboard.CenterY = (float)InteractiveSurface.PrimarySurfaceDevice.Bounds.Height - (SurfaceKeyboard.Height / 2);
SurfaceKeyboard.Layout = Microsoft.Surface.KeyboardLayout.Alphanumeric;
SurfaceKeyboard.Rotation = (float)(Math.PI / 2);
SurfaceKeyboard.ShowsFeedback = false;
}
Can someone help me please?
I don't know much about the surface framework; but usually you cannot force a keyboard to appear, the focused object needs to accept text as an input.
Because buttons generally don't accept text input, the keyboard's focus cannot be given to it, and thus
System.Windows.Input.Keyboard.Focus((IInputElement)sender);
will be ignored.
If the idea is to only make the keyboard appear, then an option is to add a SurfaceTextBox and to give focus to the textbox (this will inturn remove focus from the button)
XAML
Add this to your XAML file
<Canvas>
<s:SurfaceTextBox
Name="yourSurfaceTextBox"
Canvas.Top="200" Canvas.Left="200"
Width="100" Height="40" />
</Canvas>
Code File
void showKeyBoard(object sender, RoutedEventArgs e)
{
Console.Write("TEST");
System.Windows.Input.Keyboard.Focus((IInputElement)yourSurfaceTextBox);
// Rest of your code...
}
If the idea is to get navigation between buttons, you should consider using a SurfaceListBox since it accepts as a default behavior arrow navigation from the keyboard, then your code above should work.
Question in Comments
How I can test this on a non-surface device?
You can use a simulator which should be included in the 2.0 sdk
How I can change the cursor position in the SurfaceTextBox to the place it's touched?
I don't really understand what you mean by 'place it's touched', but you can change the cursor location in the textbox using the select method.
yourSurfaceTextBox.Select(position, 0);
To get the touch locations you can use
ReadOnlyTouchPointCollection touches = touchTarget.GetState();
Then you'll have to figure out where in relation to an object the touch was, but I this question is beyond the scope of the original question.
Have fun!

A way to monitor when a Control's screen location changes?

With WinForms, is there a way to be alerted to a control changing location with respect to the screen?
Say you have a Form with a button on it, and you would like to know when the button is moved from its current pixel location on the screen. If the button is moved to a different location on its parent Form you could obviously use the LocationChanged event, but if the Form is moved by the user, how do you know the button has visually moved?
In this simplified case the quick answer is to monitor the Form's LocationChanged and SizeChanged events, but there can be an arbitrary number of levels of nesting so monitoring those events for each parent up the chain to the primary form is not feasible. Using a timer to check if the location changed also seems like cheating (in a bad way).
Short version:
Given only an arbitrary Control object, is there a way to know when that Control's location changes on the screen, without knowledge of the control's parent hierarchy?
An illustration, by request:
Note that this "pinning" concept is an existing capability but it currently requires knowledge of the parent form and how the child control behaves; this is not the problem I am trying to solve. I would like to encapsulate this control tracking logic in an abstract Form that "pin-able" Forms can inherit from. Is there some message pump magic I can tap into to know when a control moves on the screen without having to deal with all the complicated parent tracking?
I'm not sure why you would say tracking the parent chain "is not feasible". Not only is it feasible, it's the right answer and the easy answer.
Just a quick hack at a solution:
private Control _anchorControl;
private List<Control> _parentChain = new List<Control>();
private void BuildChain()
{
foreach(var item in _parentChain)
{
item.LocationChanged -= ControlLocationChanged;
item.ParentChanged -= ControlParentChanged;
}
var current = _anchorControl;
while( current != null )
{
_parentChain.Add(current);
current = current.Parent;
}
foreach(var item in _parentChain)
{
item.LocationChanged += ControlLocationChanged;
item.ParentChanged += ControlParentChanged;
}
}
void ControlParentChanged(object sender, EventArgs e)
{
BuildChain();
ControlLocationChanged(sender, e);
}
void ControlLocationChanged(object sender, EventArgs e)
{
// Update Location of Form
if( _anchorControl.Parent != null )
{
var screenLoc = _anchorControl.Parent.PointToScreen(_anchorControl.Location);
UpdateFormLocation(screenLoc);
}
}

Handling flicks on object

I have created a new UIElement that derives from Systen.Windows.Controls.Canvas.
I am trying to handle flicks made on this object.
Everything is done in C#, in code (no XAML) using the Silverlight Phone Toolkit (February version as I want to target 7.0)
In my object constructor I do:
//Create gesture handling
gl = GestureService.GetGestureListener(this);
gl.Flick += new EventHandler<FlickGestureEventArgs>(gl_Flick);
and gl_Flick is simply:
void gl_Flick(object sender, FlickGestureEventArgs e)
{
if (e.HorizontalVelocity >= 0)
{
// Right swipe (flick)
if (gotSwipe != null)
{
gotSwipe(this, e);
}
}
}
Now, in the constructor, I also create and add a few TextBlocks
For some reason, the flick is only generating an event if done over one of those TextBocks. If I do the flick on any of the empty area of the Canvas nothing occurs.
As I can't find any documentations related to the Silverlight toolkit, everything has been done via trials&errors.
How could I do, so the flick will be recognised when performed anywhere over this canvas and not limited to over the children it contains?
I'm more focused towards WPF, but the way you talk about it reminds me of the classic null background issue.
The solution: set a non null background for your canvas.
EDIT: something like
myCanvas.Background = new SolidColorBrush(Color.FromArgb(1, 0, 0, 1));

Why does my WinForms context menu not appear where the mouse is?

In my application I have a DataGridView that is meant for configuring some options. The idea is that you can enter whatever text you want in the first column, but if you right click it will give you explicitly supported values. I need this to be a textbox rather than a dropdown list because I need to support editing invalid (or old) configurations.
What I want is the user to right click in the field name column and have a context menu that is valid based on what type of configuration this is. Therefore, I coded the following event
private void grvFieldData_CellMouseClick(object sender, DataGridViewCellMouseEventArgs e)
{
// If this is a right click on the Field name column, create a context menu
// with recognized options for that field
if (e.Button == MouseButtons.Right && grvFieldData.Columns[e.ColumnIndex].Name == "clmFieldName")
{
ContextMenu menu = new ContextMenu();
if (_supportedDataGrids.ContainsKey((cmbDataGrid.SelectedItem as DataGridFieldList).GridName))
{
// Loop through all the fields and add them to the context menu
List<string> fields = _supportedDataGrids[((cmbDataGrid.SelectedItem as DataGridFieldList).GridName)];
fields.Sort();
foreach (string field in fields)
menu.MenuItems.Add(new MenuItem(field));
// Make sure there is at least one field before displaying the context menu
if (menu.MenuItems.Count > 0)
menu.Show(this, e.Location, LeftRightAlignment.Right);
}
}
}
This works "correctly", but the context menu is appearing at the top of the form, not where the mouse pointer is. If I change the Show() call to use the DataGridView instead of the form, I have the same issue but instead it appears at the top-left hand corner of the grid, not where the mouse is.
Oddly enough, if I change this event to a MouseClick event (instead of a CellMouseclick event) everything works and the context menu appears exactly where the mouse pointer is. The problem with this option is that the user might not be right clicking on the cell that is currently selected, which means that when they click on a menu item, the selected cell will be changed and not the cell they right clicked on.
Does anyone have any hints why context menus created with the CellMouseClick are not showing at the correct spot?
menu.Show(this, e.Location, LeftRightAlignment.Right);
The 2nd argument is the mouse location, relative to the cell's upper left corner. As programmed, you make that offset relative from this, the form, which will make the menu appear in the upper left corner of the form. Use the DGV as the 1st argument doesn't work either, now it is in the upper left corner of the grid.
A couple of ways to fix this, but this is the easy way:
Point pos = this.PointToClient(Cursor.Position);
menu.Show(this, pos, LeftRightAlignment.Right);
You can arbitrarily replace this with grvFieldData.
In the datagridview mouse click event:
if e.button= mousebutton.right
{
contextmenu1.Show(MousePosition);
}
try to use PointToClient to get proper spot
It's not showing in the correct spot because e.Location is the location relative to the parent object's top-left corner, which in this case is the cell itself. Location properties are always relative to their container.
To get the position of the mouse cursor relative to the top-left of the form itself, you can use
this.PointToClient(Cursor.Position);
I have solved this problem... ones could find this method strange, but it works fine !)
If we want to see a context menu while pressing right mouse btn in the datagridview cell, and right there, not in the middle of the screen or somewhere else, we need to:
make some variables
int x=0;
int y=0;
make an 'MouseMove' Event for datagridview1 lke that:
private void dataGridView1_MouseMove(object sender, MouseEventArgs e)
{
x = e.X;
y = e.Y;
}
and
private void dataGridView1_CellMouseClick(object sender, DataGridViewCellMouseEventArgs e)
{
if (e.Button == System.Windows.Forms.MouseButtons.Right)
{
contextMenuStrip1.Show(dataGridView1, x,y);
}
}
your welcome

How to capture mousemove events beneath child controls

I am trying to handle a mouseclick event on a particular form that should fire if the mouse cursor falls between a set of coordinates - lets say a square.
I understand that if I had an empty form I could simply tie in to the mousemove event and off I go. But in reality there may be up to 10 different overlapping controls and in my test app the mousemove event only fires if the cursor is on the actual form itself and not if its over a child control.
Does anyone know how to handle this event when there are an unknown number of child controls at design time?
Is there an easy one-liner I can use?
try this:
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
AddMouseMoveHandler(this);
}
private void AddMouseMoveHandler(Control c)
{
c.MouseMove += MouseMoveHandler;
if(c.Controls.Count>0)
{
foreach (Control ct in c.Controls)
AddMouseMoveHandler(ct);
}
}
private void MouseMoveHandler(object sender, MouseEventArgs e)
{
lblXY.Text = string.Format("X: {0}, Y:{1}", e.X, e.Y);
}
}
I know this post is quite old, but it seems to me that the simplest method would be for the form to implement IMessageFilter. In the constructor (or in OnHandleCreated) you call
Application.AddMessageFilter(this);
and then you can catch the messages of all windows in your implementation of IMessageFilter.PreFilterMessage.
You'd likely need to use P/Invoke for the WIN32 IsChild method
[DllImport("user32.dll")]
public static extern bool IsChild(IntPtr hWndParent, IntPtr hWnd);
along with the form's Handle property to ensure that you're handling the right messages.
imho there is a bit of a binary situation here : and there is no "one-liner." the only solution I can see is to get your controls that don't implement events into a .NET container that does.
When any control gets a click, the normal expected behavior is that it will become the Active Control of the Form (which can always be accessed by this.ActivceControl).
But, particulary if the control you clicked captures the mouse, something has got to raise an event since .NET does not implement event "bubbling" (as WPF does).
The usual way to deal with extending behavior of any object that is sealed or whatever is to write an extension method, and I have found writing extensions for Control quite easy, but I don't know if that will help you in this case. Unfortunately I am out of my home country right now, and do not have Visual Studio to play around with.
One strategy you can use to determine if a given Point on a Form falls within the bounds of any Control is to enumerate the areas (Bounds) of all controls on the Form via 'forall of the Forms Control.Collection (this.Controls). But, if you have overlapping Controls, you then have the issue of more than one control possibly containing a given point.
best, Bill
Why don't you just use the controls' mouseover event handlers?
I know I'm a bit late to the punch, but I was having troubles with this earlier today when using a Panel as a title bar. I had a label to display some text, a picturebox and a few buttons all nested within the Panel, but I needed to trap the MouseMove event regardless.
What I decided to do was implement a recursive method handler to do this, as I only had 1 level of nested controls, this may not scale overly well when you start approaching ridiculous levels of nesting.
Here's how I did it:
protected virtual void NestedControl_Mousemove(object sender, MouseEventArgs e)
{
Control current = sender as Control;
//you will need to edit this to identify the true parent of your top-level control. As I was writing a custom UserControl, "this" was my title-bar's parent.
if (current.Parent != this)
{
// Reconstruct the args to get a correct X/Y value.
// you can ignore this if you never need to get e.X/e.Y accurately.
MouseEventArgs newArgs = new MouseEventArgs
(
e.Button,
e.Clicks,
e.X + current.Location.X,
e.Y + current.Location.Y,
e.Delta
);
NestedControl_Mousemove(current.Parent, newArgs);
}
else
{
// My "true" MouseMove handler, called at last.
TitlebarMouseMove(current, e);
}
}
//helper method to basically just ensure all the child controls subscribe to the NestedControl_MouseMove event.
protected virtual void AddNestedMouseHandler(Control root, MouseEventHandler nestedHandler)
{
root.MouseMove += new MouseEventHandler(nestedHandler);
if (root.Controls.Count > 0)
foreach (Control c in root.Controls)
AddNestedMouseHandler(c, nestedHandler);
}
And then to set it up is relatively simple:
Define your "true" handler:
protected virtual void TitlebarMouseMove(object sender, MouseEventArgs e)
{
if (e.Button == MouseButtons.Left)
{
this.Text = string.Format("({0}, {1})", e.X, e.Y);
}
}
And then set up the controls event subscribers:
//pnlDisplay is my title bar panel.
AddNestedMouseHandler(pnlDisplay, NestedControl_Mousemove);
Relatively simple to use, and I can vouch for the fact it works :)

Categories