Solve coplanar points pose estimation using EmguCV (OpenCV) - c#

I have 4 points on the same plane (a flat square object) detected in the camera and I am trying to work out the pose of this square relative to the camera.
I am using the latest version of EmguCV ( http://www.emgu.com/wiki/index.php/Main_Page ) which is a C# wrapper for OpenCV.
I have seen POSIT ( http://opencv.willowgarage.com/wiki/Posit ) but this will not work for coplanar points. I was wondering if there is anything that can solve coplanar pose estimation in OpenCV.
I have also seen solvePnp http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html#cv-solvepnp
which I believe will do what I want, but I cannot seem to find this functionality in EmguCV.
Does anyone know how to solve this using EmguCV?

Although SolvePnP is not available in emgu, you can still compute a homography once you have at least 4 point correspondences on a plane (which you have). Refer to documentation for CameraCalibration.FindHomography in case you're unsure. Once you have the homography, you can decompose this into a rotation and translation, and hence the camera pose. Take a look at this article.

Emgu.CV::CameraCalibration.SolvePnP(Emgu.CV.Structure.MCvPoint3D32f[], System.Drawing.PointF[], Emgu.CV.IntrinsicCameraParameters, Emgu.CV.CvEnum.SolvePnpMethod)
Upgrade and install the latest NuGet package.
Estimates extrinsic camera parameters using known intrinsic parameters and extrinsic parameters for each view. The coordinates of 3D object points and their correspondent 2D projections must be specified. This function also minimizes back-projection error.

Related

Constructive geometry visualization in C#

I need to visualize constructive geometry in 3d using C#. And for that I want choose a library. Ideally, that library should be able to provide the following:
Rendering arbitrary polygons in 3d. Wireframe representation.
Orhtographic projection
Camera control. I need to be able to zoom-in/out, set center of rotation to turn camera around it.
Pick my polygons, edges, vertices with a mouse.
Draw text on top of vertices, edges and polygons.
Efficency isn't the priority, so it can be resonably slow if it can do these things right out of the box or provide a good high level service for it.
I tried to look at XNA. But as far as I understood it doesn't support visualization of polygons directly.
XNA does support visualization of polygons directly by providing access to the underlying Direct3D 9 GraphicsDevice object. You can also dynamically create a mesh and draw it in a managed way, however that will be slower.
See:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/microsoft.xna.framework.graphics.graphicsdevice.drawprimitives.aspx
and related methods.
There's several DirectX wrappers in c#. I would avoid Managed DirectX since it's deprecated, and you have only access to direct3d9.
The two most common libraries for DirectX are:
SlimDX
SharpDX
Both work pretty well, and support latest DirectX feature set (SlimDX up to 11, SharpDX supports windows 8 and 11.1).
They both support what you need to do, it of course implies a bit of coding on your side.

Measuring in Kinect

I'm trying to get started with Kinect, and it has a depth sensing camera, but I've seen no guidance on measuring width/height/lengths.
Is it a matter of working out the distance an object is away from the camera (depth sensor) and at that range the field of view of the Kinect, and then working out how many pixels your object takes up?
I'd like to be able to do a mesh or something from a dot-cloud and I'm having troubles figuring where to start and how to get proper width/height measurements for objects.
This is a rather complex task and cannot be answered with a few paragraphs here on Stackoverflow. The reason is that its a lot of knowledge that builds on other knowledge. I would start by reading up on Linear Algebra using for example the excellent Rorres, et al.
Creating the mesh from the point-cloud is a complex task and there is no defacto algorithm used today. The most popular approach seems to be to first create a discretized Truncated Signed Distance Function (TSDF) and then use e.g. Marching Cubes to get a mesh. Another algorithm is the Delaunay triangulation.
There is also a c# implementation provided by the s-hull project.
In the book Beginning Kinect Programming with the Microsoft Kinect SDK by Jarrett Webb,James Ashley
you have in chapter 3 a sample of how to calculate width and height and distance :
http://books.google.es/books?id=MupB_VAmtdEC&pg=PA69&hl=es&source=gbs_toc_r&cad=4#v=onepage&q&f=false
The code is available to download at apress.com
I am aware this was asked awhile ago, but for what it's worth there is an article on channel9.msdn which shows a simpler method to calculate the user's height. I would imagine you could use a similar technique with HipLeft, HipCenter, and HipRight to estimate the width.
If you follow the project information URL from the link it has more detailed information.

Recognizing handwritten shapes

I want to recognize handwriting shape and figure out which shape it probably is in the set. Simply saying, if I draw a triangle, the application should recognize it as an triangle.
How can I do this using C# or java, any help is appreciated.
Thanks in advance.
These are some of the shapes I need to identify
You can try to use OpenCV for that. EmguCV is a good wrapper to OpenCV for .net. Watch for ShapeDetection demo (included in OpenCV)
If you want to "roll your own" I would suggest the following steps:
First, skeletonize (thin out the image till all the lines are one pixel thick). There are many ways to do this, and it is a well studied problem. Google for more information.
Now, starting at a black pixel, go through and trace out the outline of the image, one pixel at a time. You add each of these segments to a list of segments outlining the shape (each segment will be a simple line from one pixel to its adjacent pixel). Now you have the outline of your shape as a many-sided polygon.
(Possible step at this point: smooth the outline by pulling each vertex closer to the average of its neighbors)
Now, you use a corner detection algorithm to find the corners (take a look here:http://visual.ipan.sztaki.hu/corner/node7.html).
This should be enough to identify the shapes you have listed.
If you want to get smarter, you can also identify the types of edges that exist between corners. If the segment between two corners stays within some threshold of the straight line between them, you treat it as a "straight line" edge. If it doesn't, you treat it as a curving edge.
With corners +straight/curving edge, you probably could detect any of the shapes you are looking for pretty well.
I'd suggest using a neural network.
You could teach it what the shapes look like.
This is one library for example:
Neural Networks on C#
If you are looking for particular shapes inside a larger image then OpenCV is a great alternative. Emgu.CV is a good .Net wrapper for it. See my picture of a SURF implementation for this. Also see other options in OpenCV, it has plenty to offer. Note that this approach requires a lot of processing power.
If you can easily identify the shape you want as a BLOB (that is, give the algorithm a picture of only this shape) you can do a search for "ANN OCR" ("Artificial Neural Networks" and "Optical Character Recognition"). Many (most?) ANN-implementations come with sample code for feeding it shapes (letters) and recognizing closest shape (hand written letters). For example Neural Network OCR. I believe this approach would solve your problem. (Sidenote: I've encountered and tested numerous libs that can do this. It's Neural Networks 101.)
If you need BLOB algorithms for the ANN-OCR OpenCV can provide this.
Both these approaches are farily easy to implement.
There is indeed a vast tree of research in shape recognition.
If your shapes are indeed some what predictable and are basic geometry,
the most straightforward way is to find the edges and apply hough transform.
Some managable reading materials for you to start with,
[1] Google Scholar for Hough Transform Shape Detection
http://scholar.google.com/scholar?q=hough+transform+shape+recognition&hl=en&as_sdt=0&as_vis=1&oi=scholart
[2] Hough Transform # Wiki http://en.wikipedia.org/wiki/Hough_transform

C# Create "wireframe"/3D "map"

image http://prod.triplesign.com/map.jpg
How can I produce a similar output in C# window forms in the easiest way?
Is there a good library for this purpose?
I just needs to be pointed in the direction of which graphic library is best for this.
You should just roll your own in a 3d graphics library. You could use directx. If using WPF it is built-in, you can lookup viewport3d. http://msdn.microsoft.com/en-us/magazine/cc163449.aspx
In graphics programming what you are building is a very simple version of a heightmap. I think building your own would give your greater flexibility in the long run.
So a best library doesn't exist. There are plenty of them and some are just for different purposes. Here a small list of possibilities:
Tao: Make anything yourself with OpenGL
OpenTK: The successor of the Tao framework
Dundas: One of the best but quite expensive (lacks in real time performance)
Nevron: Quite good, but much cheaper (also has problems with real time data)
National Instruments: Expensive, not the best looking ones, but damn good in real time data.
... Probably someone else made some other experiences.
Checkout Microsoft Chart Controls library.
Here's how I'd implement this using OpenGL.
First up, you will need a wrapper to import the OpenGL API into C#. A bit of Googling led me to this:
CsGL - OpenGL .NET
There a few example programs available to demonstrate how the OpenGL interface works. Play around with them to get an idea of how the system works.
To implement the 3D map:
Create an array of vectors (that's not the std::vector/List type but x,y,z triplets) where x and y are along the horizontal plane and z is the up amount.
Set the Z compare to less-than-or-equal (so the overlaid line segments are visible).
Create a list of quads where the vertices of the quads are taken from the array in (1)
Calculate the colour of the quad. Use a dot-product of the quad's normal and a light source direction to get a value to shade value, i.e. normal.light of 1 is black and -1 is white.
Create a list of line segments, again from the array in (1).
Calculate the screen position of the various projected axes points.
Set up your camera and world->view transform (use the example programs to get an idea of how to do this).
Render the quads and lines, OpenGL will do the transformation from world co-ordinates (the list in (1)) to screen space. Draw the labels, you might not want to do this using OpenGL as the labels shouldn't scale with distance from camera, otherwise they could get too small to read.
Since the above is quite a lot of stuff, there isn't really the space (and time on my part) to post working code (but someone else might add something if you're lucky). You could break the task down and ask questions on the parts you don't quite understand.
Have you tried this... gigasoft data visualization tools (Its not free)
And you can checkout the online wireframe demo here

How to animate rotating cube in C#?

I would like to do something like this: a rotating cube on a form. I don't want to use any external library or dll, just pure .NET 3.5 (without directx). And a cube build with lines only. Could you please tell me how to do this?
I don't want to use external libraries because I don't need > 100 MB library to do this thing right? I want only to animate a rotating cube made with lines.
This is how you go about making a Cube in GDI+
C# 3D Drawing with GDI+ Euler Rotation
http://www.vcskicks.com/3d-graphics-improved.html
C# 3D-Drawing Cube with Shading
http://www.vcskicks.com/3d_gdiplus_drawing.html
Study assignment? This can be done with some simple 3D maths. You just need to understand the basics of matrix algebra, 3D transformations, and 3D->2D view transformation. The DirectX tutorial covers this, but you can google for it and you'll get plenty of other tutorials.
Added: Just to clarify - I'm not suggesting to use DirectX or anything. You can do this with standard System.Drawing tools. You just need to understand the math, and that's explained in the DirectX tutorials.
You might try using WPF and the 3D Tools source code released by the WPF team.
3DTools
Assuming you are using WPF for your GUI:
Make an animated PNG of the cube using a graphics program.
Use the APNG WPF Control to insert the image into your GUI.
This will yield a small assembly size and transparent background if needed.
You need a way to represent 3d points. There is no ready struct for that in .NET unless you use directx or WPF.
Then with a standard euler rotation matrix applied to the points you get the transformed points. If you only do rotations you can get away with 3x3 matrix, but if you want translation you better use 4x4 matrices and homogenous points.
After this you need a way to project those 3d points to the 2d canvas. Depending whether you are using perspective or orthographic projection the projection matrix will look a bit different.
Look into WPF in general, it will help you do this with a few measly lines of code. You can also host a WPF window in Forms.
http://msdn.microsoft.com/en-us/library/aa970268.aspx

Categories