The problem:
I've been working on trying to create a 3D position in Worldspace based on a 2D face RGB face detection, similar to this Microsoft example. I am using Unity 2020.3.16f1, MRTK 2.8.2 and C# for the Unity scripts. I have been able to convert the C++ code shown in the link to C# with a lot of success. One final issue is accessing the HoloLens 2 origin SpatialCoordinateSystem to be used in the Transform between the camera's 2D coordinate system and the 3D worldspace system.
The SO question at this link asks a very similar question, and I have tried to use SpatialLocator.GetDefault().CreateStationaryFrameOfReferenceAtCurrentLocation().CoordinateSystem() as the answers suggest. I call this function in Unity's "Awake" method, to ensure it is set as early as possible, as shown below.
private void Awake()
{
worldSpatialCoordinateSystem = SpatialLocator.GetDefault().CreateStationaryFrameOfReferenceAtCurrentLocation().CoordinateSystem;
}
Problem is that, if the user's headset is moving while the application starts, I notice an offset in the 3D locations commensurate with the direction/position of the head when the application was starting. I have narrowed the problem down to the fact that the HL2 and Unity set an origin SpatialCoordinateSystem just before the function in Awake is called, accounting for the offset between what I expect and what I see.
What I've tried:
I have tried using some of the other solutions listed here as well. I cannot use UnityEngine.Windows.WebCam.PhotoCapture becuase of the way I am create still image captures, and (SpatialCoordinateSystem)Marshal.GetObjectForIUnknown(WorldManager.GetNativeISpatialCoordinateSystemPtr()) appears to be deprecated and unusable. Finally, I tried CreateStationaryFrameOfReferenceAtCurrentLocation(Vector3, Quaternion), and used the inverse of the current Camera.main position and rotation, hoping to compensate for the offset, but it did not appear to work (NumericsConversionExtensions is the UnityEngine-to-System.Numerics converver found here). That code is below.
worldSpatialCoordinateSystem = SpatialLocator.GetDefault().CreateStationaryFrameOfReferenceAtCurrentLocation(NumericsConversionExtensions.ToSystem(Camera.main.transform.position*-1),
NumericsConversionExtensions.ToSystem(Quaternion.Inverse(Camera.main.transform.rotation))).CoordinateSystem;
My question:
Is there either another way to access the origin spatial coordinates or possibly to compensate for the offset when the user is moving their head before Awake is called?
I spent 3 days working on the solution, and found one 1 hour after asking SO. For those who come here, use the code below, originally found here.
using Microsoft.MixedReality.OpenXR;
worldSpatialCoordinateSystem = PerceptionInterop.GetSceneCoordinateSystem(Pose.identity) as SpatialCoordinateSystem;
Related
I have imported Unity Standard assets to Unity 5.6.0, I drag the 3rd person character to the scene, it keeps animating but it does not move on AWSD or Left/Right keys.
This is happening in only one project, it works on all other projects. I have searched from the net, but non of the solution seems to work.
Just try to reimport the asset with the unity third-person controller the import could have gone wrong in some place or try to build the game under build settings to see if that helped
Your character controller is made up of multiple pieces (animation files/model/script and possibly other files). There could be a problem with how you are loading the 3rd person character to the scene. I would try creating a prefab of the 3rd person character within a project and then export it as a prefab. Could be missing the script attached to the object as well. The prefab helps prevent you leaving any pieces behind that make up your character/objects. Helpful for level construction as well.
Take a look here if it helps: https://docs.unity3d.com/Manual/Prefabs.html
Another possible explanation could be how you have your input settings setup within that Unity Project. Try comparing the two inputs of the working instance, and the non-working instance.
Try looking here for a more detailed approach : https://docs.unity3d.com/Manual/class-InputManager.html
I had this problem too and in the end it was because I had my project set up to run on IOS in Build Settings. I Switched Platform to macOS and it worked.
Why? It seems that Unity's Virtual Controller expected the input to be from a Mobile Device (seems obvious now) and the CrossPlatformInputManager was not expecting WSAD.
A workaround in IOS mode is to edit the ThirdPersonCharacter/Scripts/ThirdPersonUserControl.cs and replace the ThirdPersonUserControl.cs with Input
e.g.
// read inputs
//float h = CrossPlatformInputManager.GetAxis("Horizontal");
//float v = CrossPlatformInputManager.GetAxis("Vertical");
// read inputs from default input
float h = Input.GetAxis("Horizontal");
float v = Input.GetAxis("Vertical");
I found this when I was digging around an realised the Update function in the script was getting called, however CrossPlatformInputManager.GetAxis("Horizontal") was always returning 0.
It might be worth inspecting both at runtime.
i had the same issue today, even though this thread is ancient.
for me it was because i was using my own camera and did not have the "MainCamera" tag set on it.
I use a Kinect SDK V2 sensor to capture point cloud.The output of my program is a double array that contains X-Y-Z position points for every frame(0,0,0 is the center of the sensor). Now i would like to visualize the data for example for one frame. Is any fast/easy way just to visualize the data?Any .dll?
P.S:I tried with Unity but it was to difficult and time consuming for me.I thought that code does not need it because question it too general(if code would be helpful i could upload parts).
Thank you
I am building a game that at one point relies on raw gyrometer readings. I am getting the readings using the following code (I'm using C#):
void Start () {
Input.gyrometer.enabled = true;
}
void Update() {
_guiText = Input.gyrometer.attitude.eulerAngles.ToString();
}
I'm displaying the values on screen in the OnGUI() method. As I'm using Unity I'm using a right-handed coordinate system and I've noticed that the Y value from the gyrometer corresponds to the X rotation in Unity.
The problem is that these values jump around massively. For example I can be holding the device at X:90° and rotate slowly through the Y Axis; after a certain point (around the 270° mark) the X will suddenly flip 180°. It means however, that if I start the app while the device is at 270° in the Y the X reading will be completely unusable. The same effect happens in the Z axis.
I have managed to get round this on iOS devices by resetting the gyro readings using the following snippet: Input.gyro.enabled = false; followed by Input.gyro.enabled = true; I use this to get an accurate reading of the device's absolute rotation and then I use rotational changes to get any other device movement. This doesn't work on Android however.
Has anyone come across this before? If so how did you get round it or is a bug that cannot be resolved?
This is because you are reading a conversion of Quaternion. Euler angles in Unity are just a friendly way to get a V3 representation of something that is hard to grasp. It is recommended not to affect eulerAngles directly as well as using them for reference. Instead, think of flat device as one rotation then start getting the angle between default rotation and current rotation and use that value to define your actions.
You can use Quaternion.Angle for that purpose.
ok so at the moment a camera is following the object consistently only in 1 axis. here is the code:
Matrix rotationMatrix = Matrix.CreateRotationY(avatarYaw);
Matrix rotationMatrix2 = Matrix.CreateRotationX(avatarXaw);
Vector3 transformedheadOffset2;
Vector3 transformedReference2;
transformedheadOffset2 = Vector3.Transform(AvatarHeadOffset, rotationMatrix);
transformedReference2 = Vector3.Transform(TargetOffset, rotationMatrix);
how can i make it follow the object in 2 axes? (obviously something to do with rotationMatrix2) , since when i use something like:
transformedheadOffset2 = Vector3.Transform(transformedheadOffset2 , rotationMatrix);
everything goes fuzzy. Any insight will be helpful. thanks
It is difficult to know exactly what your camera issue is. Here is a video I made to explain a common camera problem that may (or may not) be applicable to your issue.
http://www.screencast.com/users/sh8zen/folders/Xna/media/929e0a9a-16d1-498a-b777-8b3d85fd8a00
I'm not trying to just push a video I made... It's just that after 3.5 years on the xna forums, the problem that the video addresses has come up countless times from beginners working with cameras. Also, based on your description of the problem, it is very difficult to know what your camera is doing wrong so it stands a reasonable chance of being this issue.
I'm trying to use the Sprite Class in Microsoft.DirectX.Direct3D to draw some sprites to my device.
GameObject go = gameDevice.Objects[x];
SpriteDraw.Draw2D(go.ObjectTexture,
go.CenterPoint,
go.DegreeToRadian(go.Rotation),
go.Position,
Color.White);
GameObject is a class i wrote, in which all the basic information required of a game object is stored (like graphics, current game position, rotation, etc)
My dice with Sprite.Draw2D is the Position parameter (satisfied here with go.Position)
if i pass go.Position, the sprite draws at 0,0 regardless of the Position value of the Object.
i tested hard-coding in "new Point(100, 100)" and all objects drew at 100,100.
I cant figure out why the variable doesnt correctly satisfy the parameter.
I've done some googling, and many people have said MDX's Sprite.Draw2D is buggy and unstable, but i didnt find a solution.
Thus i call upon Stack Overflow to hopefully shed some light on this problem!
Fixed
Yes sprite.Draw2D some time gives problem. Have u tried sprite.Draw its working fine for me.
Here is the sample for Sprite.Draw.
GameObject go = gameDevice.Objects[x];
SpriteDraw.Draw2D(go.ObjectTexture,
new Vector3(go.CenterPoint.X,go.CenterPoint.Y,O),
new Vector3(go.Position.X,go.Position.Y,O),
Color.White); and for rotation u can use matrix transform of sprite.