Interaction, EventSystem and UI

Unity provides an great event system which can be used to interact with a scene. This solution is particularly useful for UI interfaces. InstantVR Advanced, Edge (v3.7 and up) provide extensive support for the event system using the Interaction module.

The interaction module is supported for head gaze, finger pointing and finger touching. This means that you can trigger events by looking at objects, pointing at them or by touching them. You can select which mode you want to use.

Video

Head gaze

The Head Target provides options to enable Gaze Interaction. In InstantVR you can find these in the Head Movements script.

You can enable Head Gaze by setting the Interaction parameter to Gazing.

Interaction with objects is triggered by looking at them and/or by pressing the activation button on the controller. The fields Activation Button and Controller Side determine the activation button.

Next to pressing a button, it is also possible to use a timer for activation. If the Auto Activation is set to a non-zero value the player needs to gaze at a certain object for the given numbers of seconds after which it is activated.

You can design you own focus object which will be placed at the location where the player is currently looking. The object which should be used for that is set using the Focus Point Object parameter. Note: do not put a collider on the focus point object!

Finally, the Looking at Object parameter is read-only. When the player is looking at an object, this field will contain a reference to this object.

Head gaze supports the following events:

PointerEnter when the player starts looking at an object
PointerExit when the player no longer looks at the object
PointerDown when the player presses the activation button down while looking at an object
PointerUp when the player releases the activation button while looking at an object.
PointerClick when the player has pressed and released the activation button while looking at an object
BeginDrag when the player starts to move the gaze position while pressing the activation button while looking at an object
Drag when the player moves the gaze position while pressing the activation button while looking at an object
BeginDrag when the player no longer moves the gaze position while pressing the activation button while looking at an object

Finger pointing

Events can be trigger by pointing at them. This is enabled by setting the Interaction of the hand to Pointing. This is can be set for each hand independently. In InstantVR these options are found on the Hand Movements script.

Pointing is started when the hand has the pointing pose. That is when the index finger is fully stretched and all other fingers and thumb are fully curled.

Like with head gaze, some events are triggered when an activation button is pressed. This button can be set for each hand individually using the Activation Button parameter. The controller side cannot be selected: it always matches the body side of the hand.

Is is also possible to activate by pointing at an object for a given number of seconds. The number of seconds is set using the Auto Activation parameter. When this is set to zero, auto activation is disabled.

When the Focus Point Object is set, the interaction module will put this object at the point to which the finger is pointing. If the finger is not pointing, the object will be deactivated. Additionally, if the Focus Point Object has a Line Renderer, the interaction module will update this line renderer such that it will be rendered from the finger to the position of the Focus Point Object.

The Pointing At Object is similar to the Looking At Object for the head gaze: it contains the object to which the player is pointing.

Touched Object is set while the player points at an object and had activated it.

Finger pointing supports the following events:

PointerEnter when the player starts pointing to an object
PointerExit when the player no longer pointing to the object
PointerDown when the player presses the activation button down while pointing to an object
PointerUp when the player releases the activation button while pointing to an object.
PointerClick when the player has pressed and released the activation button while pointing toan object
BeginDrag when the player starts to move the hand while pressing the activation button while pointing to an object
Drag when the player moves the hand while pressing the activation button while pointing to at an object
BeginDrag when the player no longer moves the hand while pressing the activation button while pointing to an object

Finger touch

It is also possible to interact with objects with your hands directly by touching them with the index finger. This is enabled by setting the Interaction to Touching in the Hand Movements.

Pointing At Object is not set in Touching mode.

Touched Object is set when the player is actually touching the object.

Finger pointing supports the following events:

PointerDown when the player presses on an object with the index finger
PointerUp when the player no longer presses on an object with the index finger
PointerClick when the player has pressed and released on an object with the index finger
BeginDrag when the player starts to move the index finger while pressing on an object
Drag when the player moves the index finger while pressing an object
BeginDrag when the player no longer moves the index finger while pressing an object