I am working on a project which has following goal:
Load rigged 3D mesh (e.g. a human skeleton) with Assimp.NET
Manipulate bones of mesh so it fits your own body (with Microsoft Kinect v2)
Perform vertex skinning
Loading the rigged mesh and extracting bone information works (hopefully) without any problems (based on this tutorial: http://www.richardssoftware.net/2013/10/skinned-models-in-directx-11-with.html). Each bone (class "ModelBone") consists of following information:
Assimp.Matrix4x4 LocalTransform
Assimp.Matrix4x4 GlobalTransform
Assimp.Matrix4x4 Offset
LocalTransform is directly extracted from assimp node (node.Transform).
GlobalTransform includes own LocalTransform and all parent's LocalTransform (see code snipped calculateGlobalTransformation()).
Offset is directly extracted from assimp bone (bone.OffsetMatrix).
At the moment I don't have GPU vertex skinning implemented, but I iterate over each vertex and manipulate it's position and normal vector.
foreach (Vertex vertex in this.Vertices)
{
Vector3D newPosition = new Vector3D();
Vector3D newNormal = new Vector3D();
for (int i=0; i < vertex.boneIndices.Length; i++)
{
int boneIndex = vertex.boneIndices[i];
float boneWeight = vertex.boneWeights[i];
ModelBone bone = this.BoneHierarchy.Bones[boneIndex];
Matrix4x4 finalTransform = bone.GlobalTransform * bone.Offset;
// Calculate new vertex position and normal
newPosition += boneWeight * (finalTransform * vertex.originalPosition);
newNormal += boneWeight * (finalTransform * vertex.originalNormal);
}
// Apply new vertex position and normal
vertex.position = newPosition;
vertex.normal = newNormal;
}
Like I already said, I want to manipulate bones with a Kinect v2 sensor, so I won't have to use animations (e.g. interpolating keyframes, ...)! But for the beginning I want to be able to manipulate bones manually (e.g. rotate torso of mesh by 90 degrees). Therefore I create a 4x4 rotation matrix (90 degrees around x-axis) by calling Assimp.Matrix4x4.FromRotationX(1.5708f);. Then I replace the bone's LocalTransform with this rotation matrix:
Assimp.Matrix4x4 rotation = Assimp.Matrix4x4.FromRotationX(1.5708f);
bone.LocalTransform = rotation;
UpdateTransformations(bone);
After the bone manipulation I use following code to calculate the new GlobalTransform of the bone and it's child bones:
public void UpdateTransformations(ModelBone bone)
{
this.calculateGlobalTransformation(bone);
foreach (var child in bone.Children)
{
UpdateTransformations(child);
}
}
private void calculateGlobalTransformation(ModelBone bone)
{
// Global transformation includes own local transformation ...
bone.GlobalTransform = bone.LocalTransform;
ModelBone parent = bone.Parent;
while (parent != null)
{
// ... and all local transformations of the parent bones (recursively)
bone.GlobalTransform = parent.LocalTransform * bone.GlobalTransform;
parent = parent.Parent;
}
}
This approach results in this image. The transformation seems to be applied correctly to all child bones, but the manipulated bone rotates around the world space origin and not around its own local space :( I already tried to include the GlobalTransform translation (last row of GlobalTransform) into the rotation matrix before set it as the LocalTransform, but without success...
I hope somebody can help me with this problem!
Thanks in advance!
Finally I found the solution :) All calculations were correct except:
Matrix4x4 finalTransform = bone.GlobalTransform * bone.Offset;
The correct calculation for me is:
Matrix4x4 finalTransform = bone.GlobalTransform * bone.Offset;
finalTransform.transpose();
So it seems to be a row-major / column-major problem. My final CPU vertex skinning code is:
public void PerformSmoothVertexSkinning()
{
// Precompute final transformation matrix for each bone
List<Matrix4x4> FinalTransforms = new List<Matrix4x4>();
foreach (ModelBone bone in this.BoneHierarchy.Bones)
{
// Multiplying a vector (e.g. vertex position/normal) by finalTransform will (from right to left):
// 1. transform the vector from mesh space to bone space (by bone.Offset)
// 2. transform the vector from bone space to world space (by bone.GlobalTransform)
Matrix4x4 finalTransform = bone.GlobalTransform * bone.Offset;
finalTransform.Transpose();
FinalTransforms.Add(finalTransform);
}
foreach (Submesh submesh in this.Submeshes)
{
foreach (Vertex vertex in submesh.Vertices)
{
Vector3D newPosition = new Vector3D();
Vector3D newNormal = new Vector3D();
for (int i = 0; i < vertex.BoneIndices.Length; i++)
{
int boneIndex = vertex.BoneIndices[i];
float boneWeight = vertex.BoneWeights[i];
// Get final transformation matrix to transform each vertex position
Matrix4x4 finalVertexTransform = FinalTransforms[boneIndex];
// Get final transformation matrix to transform each vertex normal (has to be inverted and transposed!)
Matrix4x4 finalNormalTransform = FinalTransforms[boneIndex];
finalNormalTransform.Inverse();
finalNormalTransform.Transpose();
// Calculate new vertex position and normal (average of influencing bones)
// Formula: newPosition += boneWeight * (finalVertexTransform * vertex.OriginalPosition);
// += boneWeight * (bone.GlobalTransform * bone.Offset * vertex.OriginalPosition);
// From right to left:
// 1. Transform vertex position from mesh space to bone space (by bone.Offset)
// 2. Transform vertex position from bone space to world space (by bone.GlobalTransform)
// 3. Apply bone weight
newPosition += boneWeight * (finalVertexTransform * vertex.OriginalPosition);
newNormal += boneWeight * (finalNormalTransform * vertex.OriginalNormal);
}
// Apply new vertex position and normal
vertex.Position = newPosition;
vertex.Normal = newNormal;
}
}
}
Hopefully this thread is helpful for other people. Thanks for your help Sergey!
To transform a bone you should use it's offset matrix:
http://assimp.sourceforge.net/lib_html/structai_bone.html#a9ae5293b5c937436e4b338e20221cc2e
Offset matrix transforms from global space to bone space. If you want to rotate a bone around it's origin you should:
transform to bone space
apply rotation
transform to global space
So bone's global transform may be calculated like this:
bonesGlobalTransform = parentGlobalTransform *
bone.offset.inverse() *
boneLocalTransform *
bone.offset;
So:
transform to bone space with offset matrix
apply local transform
transform to global space with offset.inverse() matrix
Related
I'm having a problem in Monogame/XNA where I am storing the position/rotation/scale in a matrix for easier manipulation of these values with parenting. It all seems to be going well, except for when I rotate the matrix, which is when the object seems to jitter around and shake. Here is the relevant code:
// Executes every frame
internal void Draw()
{
float RadRot = MathHelper.ToRadians(Rotation);
// Sets world matrix for things that draw
if (Parent != null) {
WorldMatrix = Matrix.CreateScale(Size.X, Size.Y, 1) * -Matrix.CreateTranslation(Pivot.X, Pivot.Y, 0) * Matrix.CreateRotationZ(RadRot);
WorldMatrix.Translation = Vector3.Zero; // Rotating seems to create a strange positioning issue, this fixes it
WorldMatrix *= Matrix.CreateTranslation(RelativePosition.X, RelativePosition.Y, 0) * Matrix.CreateRotationZ(MathHelper.ToRadians(Parent.Rotation)) * <FIX JITTER> * Matrix.CreateTranslation(Parent.DecomposedPosition.X, Parent.DecomposedPosition.Y, 0);
} else
{
// For objects without parents
WorldMatrix = Matrix.CreateScale(RelativeSize.X, RelativeSize.Y, 1) * -Matrix.CreateTranslation(Pivot.X, Pivot.Y, 0) * Matrix.CreateRotationZ(RadRot);
WorldMatrix.Translation = Vector3.Zero; // This fixes the jitter issue, but in the code above (in the if statement), I can't do this when translating the child around the parent
WorldMatrix *= Matrix.CreateTranslation(RelativePosition.X, RelativePosition.Y, 0);
}
Vector3 TempPos = Vector3.Zero;
Quaternion TempRot = Quaternion.Identity;
Vector3 TempSize = Vector3.Zero;
WorldMatrix.Decompose(out TempSize, out TempRot, out TempPos);
DecomposedPosition = new Vector2(-TempPos.X, -TempPos.Y);
DecomposedRotation = MathHelper.ToRadians(Rotation); // Can't convert quaternions, but this works
DecomposedSize = new Vector2(MathF.Round(TempSize.X), MathF.Round(TempSize.Y));
foreach (Component2D comp in Components)
{
comp.Draw();
}
}
This is what comp.Draw calls:
MyGame.SpriteDrawing.Draw(Image, new Rectangle(LinkedObject.DecomposedPosition.ToPoint(), LinkedObject.DecomposedSize.ToPoint()), RenderRegion, Color, LinkedObject.DecomposedRotation, LinkedObject.Pivot, SpriteEffects.None, LinkedObject.Layer + (SubLayer / 10000));
Here is what it looks like in action:
EXTRA NOTE: This jittering does occur no matter the scale of the objects
Give it a try to organize the scene in a different manner:
Each object let store its scale (Vector3), rotation (Quaternion), and position (Vector3)
Each object should have a property of result matrix (in local space), which will be the multiplication of something like this in following order: Matrix.CreateScale(scale) * Matrix.CreateRotationQuaterion(rotation) * Matrix.CreateTranslation(position)
Now, when you have the above local space Matrix, use it to create WorldMatrix, which is a multiplication of Parent World Matrix and this object local space Matrix, or local space Matrix only - when no Parent is present.
Try to pass only the world matrix of each element to your drawing code, avoid decomposition
If you have a large hierarchy, you could cache local matrix results if it is not changed
It seems that I was simply using the wrong SpriteBatch.Draw(). I was using the Rectangle version, whereas all I needed was to use the Vector2 version to prevent rounding errors.
MyGame.SpriteDrawing.Draw(Image, LinkedObject.DecomposedPosition, RenderRegion, Color, LinkedObject.DecomposedRotation, LinkedObject.Pivot * RenderRegion.Size.ToVector2(), LinkedObject.DecomposedSize / RenderRegion.Size.ToVector2(), SpriteEffects.None, LinkedObject.Layer + (SubLayer / 10000));
So, here is what i have:
SteamVR's hand gameobject
3D sphere.
what i want:
The sphere to move to same direction/position as the hand does, but it to move further with a multiplier. E.g. i move the VR controller and the hand moves 1 unit. I want that the sphere moves to the same direction in a same amount of time but e.g. 2 units. how do i do this?
i tried simple
sphere.transform.position = ControllerToFollow.position +2f;
but then the sphere is always offset.
position is a Vector3, which is essentially 3 floats - you can't plus a Vector3 with a float unless you overload the + operator. Otherwise what you can do is the following:
Vector3 followPos = new Vector3(ControllerToFollow.position.x + 2f,
ControllerToFollow.position.y + 2f,
ControllerToFollow.position.z + 2f);
sphere.transform.position = followPos;
If you only want it to follow on one axis, then you can do the following:
Vector3 followPos = new Vector3(ControllerToFollow.position.x + 2f, // Follow on x Axis
ControllerToFollow.position.y, // Y axis is the same
ControllerToFollow.position.z); // X Axis is the same
sphere.transform.position = followPos;
Edit: I think I understand your problem better now. Here's a better version.
if (Vector3.Distance(sphere.transform.position, ControllerToFollow.position) >= 2f)
{
// Code that makes the sphere follow the controlling
}
Just track the movement Delta of the hand and multiply it by a certain multiplier.
At the beginning of the manipulation store
private Vector3 lastControllerPosition;
...
lastControllerPosition = ControllerToFollow.position;
then in every frame compare
var delta = ControllerToFollow.position - lastHandPosition;
// Don't forget to update lastControllerPosition for the next frame
lastControllerPosition = ControllerToFollow.position;
Now in delta you have a movement of the controller since the last frame. So you can assign it to the sphere with a multiplier using Transform.Translate
sphere.transform.Translate(delta * multiplier, Space.World);
or simply using
sphere.transform.position += delta * multiplier;
Example Image here
I am trying to find a way to calculate points on my cylinders top circle surface. My situation looks like this, I have a vector which is defining my cylinders direction in 3d room. Then I already calculated me a perpendicular vector with
Vector3.Cross(vector1, vector2)
Now I use the diameter/2 to calculate the point which is lying on the edge of the circular top surface of my cylinder. Now I want to rotate my vector always 90 degrees in order to get 4 points on the edge of the surface. All the 4 vectors defining them should be perpendicular to the cylinders direction. Can you help me how I can rotate the first perpendicular to achieve this?
I already tried:
Matrix4x4.CreateFromAxisAngle(vectorcylinderdirection, radiant)
Then I calculated again cross product but it doesnt work like I want to.
Edit:
public static void calculatePontsOnCylinder()
{
//Calculate Orthogonal Vector to Direction
Vector3 tCylinderDirection = new Vector3(1, 0, 0);
Vector3 tOrthogonal = Vector3.Cross(tCylinderDirection, new Vector3(-tCylinderDirection.Z,tCylinderDirection.X,tCylinderDirection.Y));
Vector3 tNormOrthogonal = Vector3.Normalize(tOrthogonal);
//Calculate point on surface circle of cylinder
//10mm radius
int tRadius = 10;
Vector3 tPointFinder = tNormOrthogonal * tRadius;
//tPointFinder add the cylinder start point
//not yet implemented
//now i need to rotate the vector always 90 degrees to find the 3 other points on the circular top surface of the cylinder
//don't know how to do this
// I thought this should do it
Matrix4x4.CreateFromAxisAngle(tCylinderDirection, (float)DegreeToRadian(90));
}
private static double DegreeToRadian(double angle)
{
return Math.PI * angle / 180.0;
}
In the picture you can see a example, the vector1 is what I need, always rotated 90 degrees and vector2 would be my cylinder direction vector
I possibly have found the correct formula:
Vector3 tFinal = Vector3.Multiply((float)Math.Cos(DegreeToRadian(90)), tPointFinder) + Vector3.Multiply((float)Math.Sin(DegreeToRadian(90)), Vector3.Cross(tCylinderDirection, tPointFinder));
Vector3 tFinal180 = Vector3.Multiply((float)Math.Cos(DegreeToRadian(180)), tPointFinder) + Vector3.Multiply((float)Math.Sin(DegreeToRadian(180)), Vector3.Cross(tCylinderDirection, tPointFinder));
Vector3 tFinal270= Vector3.Multiply((float)Math.Cos(DegreeToRadian(270)), tPointFinder) + Vector3.Multiply((float)Math.Sin(DegreeToRadian(270)), Vector3.Cross(tCylinderDirection, tPointFinder));
Interesting is that if I try it with (1,1,0) as cylinder direction it gives me correct directions but the length is different for 90 degrees and 270.
Here is the code that should solve your problem assuming that the input requirements are satisfied.
float zCutPlaneLocation = 20; // should not get bigger than cylinder length
float cylinderRadius = 100;
Vector3 cylinderCenter = new Vector3(0, 0, 0); // or whatever you got as cylinder center point, given as Vector3 since Point type is not defined
// will return 360 points on cylinder edge, corresponding to this z section (cut plane),
// another z section will give another 360 points and so on
List<Vector3> cylinderRotatedPointsIn3D = new List<Vector3>();
for (int angleToRotate = 0; angleToRotate < 360; angleToRotate++)
{
cylinderRotatedPointsIn3D.Add(GetRotatedPoint(zCutPlaneLocation, angleToRotate, cylinderRadius, cylinderCenter));
}
....
private static Vector3 GetRotatedPoint(
float zLocation, double rotationAngleInRadian, float cylinderRadius, Vector3 cylinderCenter)
{
Vector2 cylinderCenterInSection = new Vector2(cylinderCenter.X, cylinderCenter.Y);
float xOfRotatedPoint = cylinderRadius * (float)Math.Cos(rotationAngleInRadian);
float yOfRotatedPoint = cylinderRadius * (float)Math.Sin(rotationAngleInRadian);
Vector2 rotatedVector = new Vector2(xOfRotatedPoint, yOfRotatedPoint);
Vector2 rotatedSectionPointOnCylinder = rotatedVector + cylinderCenterInSection;
Vector3 rotatedPointOnCylinderIn3D = new Vector3(
rotatedSectionPointOnCylinder.X,
rotatedSectionPointOnCylinder.Y,
zLocation + cylinderCenter.Z);
return rotatedPointOnCylinderIn3D;
}
I just created a console app for this. First part of code should be added in main method.
Working with those matrices seems is not that easy. Also I am not sure if your solution works ok for any kind of angle.
Here the idea is that the rotated points from cylinder are calculated in a section of the cylinder so in 2D than the result is moved in 3D by just adding the z where the Z section was made on cylinder. I suppose that world axis and cylinder axis are on the same directions. Also if your cylinder gets along (increases) on the X axis, instead of Z axis as in example just switch in code the Z with X.
I attached also a picture for more details. This should work if you have the cylinder center, radius, rotation angle and you know the length of the cylinder so that you create valid Z sections on cylinder. This could get tricky for clockwise/counter clock wise cases but lets see how it works for you.
If you want to handle this with matrices or whatever else I think that you will end up having this kind of result. So I think that you cannot have "all" the rotated points in just a list for the entire cylinder surface, they would depend on something like the rotated points of a Z section on the cylinder.
I am writing a custom script for exporting Blender models and animation into a game that runs using OpenGL.
I am aware that Blender uses a Z-up system whereas OpenGL in my game uses a Y-up system. I can easily fix the model mesh and bone positions with a rotation around the X axis of -π/2, which renders the model correctly.
However in Blender a bone on my character's leg that is parallel with the floor has an euler X rotation of 0, and a bone that is perpendicular to the floor has an euler X rotation of -π/2 (or π/2).
In-game, a character's leg that is parallel to the floor has an euler X rotation of -π/2 (or π/2) and a bone that is perpendicular to the floor has a rotation of 0.
Here, the model's left leg is parallel with the floor with the same XYZ euler rotation as the left leg in Blender (which is perpendicular to the floor):
The blender export script for bone rotations at each frame is:
for f in range(scene.frame_start, scene.frame_end + 1):
bpy.context.scene.frame_set(f);
bpy.context.scene.update();
for i in range(len(arm.pose.bones)):
bone = arm.pose.bones[i]
out.write(str(boneIDs[bone.name]) + ","); # ID of the bone
out.write(VecToStringComma(bone.matrix.to_euler())) # bone rotation
out.write("\n")
And loading it in C# the bone rotation matrix is recomposed with the following code:
for (int i = 0; i <= frameCount; i++)
{
Frame frame = new Frame();
var parts = r[read].Split(',');
read++;
for (int p = 0; p < parts.Length - 1; p += 4)
{
FrameDeform fd = new FrameDeform();
fd.DeformerID = int.Parse(parts[p]);
Vector3 rot = new Vector3(double.Parse(parts[p + 1]), double.Parse(parts[p + 2]), double.Parse(parts[p + 3]));
fd.Rotation = Matrix4.CreateRotationX(rot.X) * Matrix4.CreateRotationY(rot.Y) * Matrix4.CreateRotationZ(rot.Z);
frame.FrameDeforms.Add(fd);
}
anim.Frames.Add(frame);
}
Frame 0 in Blender looks like this:
Frame 0 in-game looks like this:
I am aware that at the moment I am not applying any alterations to the rotation values, however I can see that the legs are following the right movement path in-game, but on the incorrect axis.
Rotating the rotation values by -π/2 around the X axis does not help, as the X values remain unchanged.
Any help would be incredibly appreciated as I have been struggling with this issue for quite a while.
Update
When loading the model I apply no rotations around the X axis. Instead, at run-time I apply a rotation of -π/2 around the X axis, which happens in the shader:
vec3 pos = (zMat * bones[id] * vec4(aPosition, 1.0)).xyz
However this gives me the following result (when only animating the torso):
I am getting 2d coordinates from Vector3. I have to correct positions to get right results. Indeed it seems I get correct positions like this but I do not know how to correct position when rotated.
Here my worldViewMatrix that I do operations but those operations not passed to my VertexData then I try to correct positions.
WorldViewMatrix = Matrix.Scaling(Scale) * Matrix.RotationX(Rotation.X) * Matrix.RotationY(Rotation.Y) * Matrix.RotationZ(Rotation.Z) * Matrix.Translation(Position.X, Position.Y, Position.Z) * viewProj;
I am trying to correct it like:
public Vector2 Convert_3Dto2D(Vector3 position, Vector3 translation, Vector3 scale, Vector3 rotation, Matrix viewProj, RenderForm_EX form)
{position += translation;
position += translation;
position = Vector3.Multiply(position, scale);
//ROTATION ?
var project = Vector3.Project(position, 0, 0, form.ClientSize.Width, form.ClientSize.Height, 0, 1, viewProj);
Console.WriteLine(project.X+" "+ project.Y);
return new Vector2(project.X, project.Y);
}
What can I do to correct rotated position ?
If you can, post a little more information about "correct positions". I will take a stab at this and assume you want to move your vertex into world space, then work out what pixel it occupies.
Usually you order multiplying your order by
Translate * Rotate * Scale;
if you want Viewprojection to apply correctly, I believe it should be at the start. V * (t * r * s).
The following link on gamedev stackexchange goes into this. matrix order
Also, your project takes in a Vector3 that has been already multiplied into wvp matrix, I dont see you have multiplied it in your convert_3dto2d function.
Basically, execute a TRS matrix multiply on your original vert, then multiply your WVP matrix then execute your project. You will then get your screen space pixel.