Using Humanoid Control with VRTK

VRTK for Unity has become a very popular package for creating virtual reality applications. Unfortunately, it is not directly compatible with Humanoid Control. In this post we will explain how you can use the two together to get the best out of this combination.

The issue

On of the starting points of Humanoid Control is that it determines at run-time which body tracking hardware is available. It will then use the correct SDKs to support each device. This makes it possible to make an executable which works with many devices. In VRTK you have to select the SDK at design time. This makes it hard to use the two together.

The solution

The best solution is to use VRTK for all tracking input and use Humanoid Control to do the avatar animation.

In this setup, we do not use any tracking extension of Humanoid Control itself. For this reason you should disable all tracking extensions listed in the Preferences. You can find these in the Edit Menu->Preferences->Humanoid. We will be using VRTK for tracking with SteamVR, Oculus or other devices.

Because of the same reason we need to disable First Person Camera on the Humanoid Control script too: this will make sure that no camera is created on the head by Humanoid Control: we will be using the camera of VRTK.

Now you can setup VRTK in the way you are used to. For this example though we are going to use SteamVR and the [CameraRig] prefab.

Matching the Targets to VRTK

What we need to do now is connect the targets of the humanoid to the correct transform in the VRTK hierarchy. In order to be able to do this, we need to create the targets in the hierarchy. You can do this by expanding the Targets section and click Show for all applicable targets. In our example we need the Head Target and Hand Targets to be available in the hierarchy.

Showing Targets in the Inspector

Showing Targets in the Hierarchy

Head Target

We need to match the Camera (head) transform within the [CameraRig]  to the Head Target of Humanoid Control. We do this in the following way:

  1. Drag the Camera (head) in the hierarchy such that it becomes a child of the Head Target.
  2. Set all position and rotation values of the Camera (head) to zero. This will make sure that the camera has the same location and orientation as the Head Target.
  3. Now you need to move the Camera (head) to the position precisely between the eyes of the avatar. With the default Humanoid avatar, this position will be (0, 0.13, 0.13) relatively to the Head Target. For other avatars this may be different
  4. Now that the Camera (head) has the correct position, we can drag it back to its original place in the hierarchy: as a child of the [CameraRig].
  5. Finally, we need to drag the Head Target to be a child of the Camera (head). This will make sure that the Head Target will be moved by the SteamVR camera. And as moving the Head Target will move the head of the avatar, we have reached our goal.

So the resulting hierarchy at the [CameraRig] is now as below, with the local position of the Head Target (0, -0.13, -0.13). The Head Target should be on the neck of the avatar now, the Camera (head) between the eyes of the avatar.

The Head Target in the VRTK Hierarchy

Hand Targets

The Hand Targets are a bit harder initially because we cannot see the SteamVR controllers in the scene view when the scene is not playing. This means that we cannot check whether the hands of the avatar will be holding the controllers in the correct position.

We can solve this by adding models of the SteamVR Controllers manually. You can remove them later again. When you have included SteamVR support for Humanoid Control in your project, you can find these models at the following location Assets/Humanoid/Scripts/Extensions/SteamVR/Resources/Vive Controller. Drag this prefab into your scene as a child of the Controller (left) with local position and local rotation being (0,0,0). Now we follow a similar process as with the Head Target.

  1. Drag the Controller (left) in the hierarchy such that it becomes a child of the Left Hand Target.
  2. Set all position and rotation values of the Controller (left) to zero. This will make sure that the controller will have the same location and orientation as the Hand Target.
  3. Now you need to move the Controller (left) to a position where the hand is holding the Vive Controller mesh in a neat way. With the default avatar, the position will be (-0.14, -0.04, 0.08) and the rotation (0, -30, -90) relatively to the Hand Target. For other avatars this may be different.
  4. Now that the controller has the correct position and orientation, we can drag it back to its original place in the hierarchy: as a child of the [CameraRig].
  5. Finally, we need to drag the Left Hand Target such that it becomes a child of the Controller (left). This will make sure that the Left Hand Target will be moved by the left SteamVR Controller. And as moving a Hand Target will move a hand of the avatar, we reached our goal for the left hand.

Now the resulting hierarchy at the [CameraRig] is now as below, with the local position of the Left Hand Target (-0.04,0.08,-0.14) and the rotation being(-30,0,90). The Left Hand Target should be on the left wrist of the avatar now, the Controller (left) should be in the left hand of the avatar.

The Left Hand Target in the VRTK Hierarchy

Of course you will need to do the same for the right hand. Except in stap 3, the local position with the default avatar should be (0.14, -0.04, 0.08) and local rotation (0, 30, 90).

Conclusion

That is it! You may remove the Vive Controller models we added to the hands now, because the SteamVR_RenderModel script on the Model objects will show them while playing the game anyway.

When pressing play, the avatar should follow the movements of the headset and controllers now.

Posted in HowTo, Humanoid Control
2 comments on “Using Humanoid Control with VRTK
  1. DAG says:

    one of the things i was waiting for Really happy to see this it replaces VRIK ( Final IK works good for AI, just not VR players) issues

    your VR controls Works grate…. i just like the ease of VRTK.

    last thing i would need to learn is how to add an Avatar at run time for UMA – Unity Multipurpose Avatar ( run time generated Avatar). that would be golden,

    • Thanks for your kind words.
      UMA support is on my to-do list for Humanoid Control. It already recognizes the avatars rig, but direct support for these avatars requires more work.
      I already have had contact with the UMA team and they were willing to cooperate on this. Currently I am very busy launching Humanoid Control, but when that has settled I will work on full UMA support.

Leave a Reply