VRTK Hand Poses

Hello, everyone. Welcome to lecture two of the course where you will learn to set up hand poses for your weapons and ammo to make grabbing look truly realistic. You will also create a new layer for interactable objects and assign all four interactable objects to this layer. You will also go on to set up the layer collision matrix for physics interactions between interactable objects assigned to the same layer. Let’s begin.

VRTK provides you with the ability to set up hand poses for each hand that can grab a particular interactable object. Hand posing allows you the ability to control the amount of curl your fingers can express onto an object when grabbed. Using hand posing, you can curl your fingers to the exact extent required to ensure that they don’t clip into the object’s geometry, thus making the grab look truly realistic.

When you grab your M48 with your left hand, you will set up a hand pose for a left-hand grab against the M48. Likewise, when you grab the M48 with your right hand, you will need to set up a hand pose for a right-hand grab against it. This would apply to each interactable object in your scene where you would set up a hand pose for each interactable object for each hand.

Let’s first set up the hand pose for the M48 when your right hand grabs it. In the Hierarchy, select the VRTK_SETUP game object and create a new Empty Child game object within it. Rename it to VRTK_HAND_POSES. All hand poses you create for your interactable objects will reside as children of this game object.

From within the Project tab, expand the Packages folder and locate the Tilia package, Tilia Visuals BasicHand Unity. Expand this folder. Then expand its Runtime folder and select the Prefabs folder. You will note that it contains a prefab called Visuals.BasicHandPoser.

At the time of this recording, this prefab is not part of the Tilia prefab’s visuals avatar hands context menu item. So the only way to access it currently is via the packages folder. In the Hierarchy with the VRTK_HAND_POSES game object selected, drag and drop the Visuals.BasicHandsPoser prefab onto it. Rename this prefab, M48_HAND_POSER.

With the M48_HAND_POSER selected in the Hierarchy, expand it. You will note that it contains two game objects, namely a Left.BasicHandPoser and a Right.BasicHandPoser, which allow you to create hand poses for your left and right hands.

In the hierarchy, expand the camera rigs tracked alias, further expand aliases, and then expand the left and right controller alias. Now expand the left interactor and then the avatar container game object within it. Also, expand the right interactor and then the avatar container game object within it. Now expand the M48_HAND_POSER prefab and further expand its Right.BasicHandPoser game object.

Select the M48_HAND_POSER prefab in the Hierarchy. In the Inspector, within its Hand Poser Facade component, you will note that it contains two properties, namely left-hand container and right-hand container that need to be populated. From the hierarchy, drag and drop the left interactor as the value for the left-hand container property.

Next, drag and drop the right interactor as the value for the Right Hand Container property. Next select the Right.BasicHandPoser game object in the hierarchy. Drag and drop the Right.BasicHand onto its target property. Within the Right.BasicHandPoser, you will note that its child objects are the fingers of your hand. You will set up a pose for when a grab occurs.

That is where your middle, ring, and pinky fingers curl in to depict a grab. When you set this up for your hands in general, the middle ring and pinky fingers curled all the way in when the grip button was pressed on the controller. However, to have this pose look more realistic when grabbing the M48, you will have these three fingers curl in 75% of the way only. Ideally, you would want to have these fingers curl in just a bit till they are touching the M48’s handle grip without actually clipping into its handle grip.

In the hierarchy, select the middle, ring, and pinky finger pose game objects. In the Inspector, within the linked Finger Poser component, for the Input Source property, select Override from the dropdown as the source input is going to be the override value you select next. Now set the override value to 0.75 to curl these fingers approximately 75% of the way. You can tweak this value later once you have play tested against your VR device.

Now you need to select values from the dropdown for the Apply Pose Values property, letting it know what property values you have set so that the same may be applied to the selected fingers at the time the pose takes effect. In this case, you have changed two property values. That is Input Source and Override Value.

For the Apply Pose Values property, from its dropdown, select the item’s input source and override value. Now from within the hierarchy, select the Interactions interactable M48 weapon prefab. In the Inspector within its Grab Event section, available within the Interactable Facade Component, locate and expand the events Grabbed and Ungrabbed.

The Grabbed event is called each time you grab your M48 weapon. Similarly, the Ungrabbed event gets called the moment you stop grabbing your M48 weapon. When your M48 is grabbed and its grabbed event is invoked, you would ideally want to perform some action. In the case when the M48 is grabbed, the grab pose you have set up should be applied, and upon being ungrabbed, the grab pose should return to its normal ungrabbing stance. Let’s set up these events now.

Ensure that the Grabbed and Ungrabbed events have been expanded in the inspector. Click the plus icon located at the bottom of these events to add an event listener for each event. Now from the hierarchy, drag and drop the M48_HAND_POSER prefab onto the event box for each of these events.

Now for the Grabbed event, click on the dropdown that lists no function and select HandPoserFacade. Apply Pose available within the Dynamic Interactive Facade section. Now whenever your M48 is grabbed, its grabbed event is triggered, and the hand pose you have set up will be applied.

Now for the Ungrabbed event, click on the dropdown that lists no function and select HandPoserFacade RemovePose, available within the Dynamic Interactor Facade section. Now whenever you ungrab your M48, it’s Ungrabbed event is triggered and the hand pose will return back to its normal stance.

Let’s now set up the pose for your index finger wherein your index finger is used to pull on the trigger of a weapon. Ideally, you will want to curl this finger in too, but not to its full extent, and probably just enough to provide a realistic visual cue indicating that you are squeezing the trigger of your weapon. Let’s begin.

In the hierarchy, select the index finger pose available within the Right.BasicHandPoser game object. When setting up your hand’s index finger, you had set a value for the max curl limit. You will be taking a similar approach here. Set the Max Curl Limit value to 0.5.

Set the Force Transition Threshold value to a value smaller than 0.5. You could set this value to 0.45. You can also set the transition speed to a smaller value such as 0.1 should you want a faster curl. However, let’s leave it at its default value of 0.3.

For the Apply Pose Values property, from its dropdown, select the items CurlLimits and ForceTransitionThreshold, as these are the values you have changed. Should you decide you want a faster transition speed and change the value of the transition property to, say, 0.1, then ensure you add it to the Apply Pose Values property, too.

Lastly, you may want to set up the thumb to curl by default to present an overall realistic curled hand grab. Let’s set this up. In the Hierarchy, select the thumb pose available within the Right.BasicHandPoser game object.

For the Input Source, select the item override from the dropdown. Set the Override Value property to 0.6 to have your thumb curl in 60% of the way. Finally, for the Apply Pose values property, select the item’s Input Source and Override Value from the dropdown.

Now you have set up a hand pose for your right hand. Mount your VR headset and play test your scene. Once your scene has loaded up within your VR headset, prior to grabbing your M48 press the grab button on your right controller and note that your middle, ring, and pinky fingers curl in completely. Now press the trigger button on your right controller and note how your index finger curls in completely. Finally, note that your thumb is in its extended position currently. Press the thumb stick on your right controller and note how your thumb curls in.

Now grab your M48 with your right hand and notice how the hand pose takes effect. You will note that the middle ring and pinky fingers now don’t curl in all the way. They curl in only to the extent where it looks like you have grabbed the weapon, around its handle grip without clipping into it. You can tweak the value of 0.75 that you set to get the exact feel you desire.

Now pull down fully on your right controller’s trigger using your index finger and note how the hand pose you set up for the index finger curls in on the trigger. Lastly, note that your thumb has curled in by default, and you don’t need to press on the thumb stick to have this curl take effect. Try out combinations of finger presses to see the hand pose curl take effect. You would most likely need to fine tune the values that have been set up by playtesting each finger’s hand pose in VR and tweak the property values within the linked Finger Poser component for each group of fingers.

As your first challenge, set up the hand pose for your left hand. Within the M48 HandPoser prefab, you will need to expand the Left.BasicHandPoser game object, and then set up the fingers with similar values as was done for the right hand. Once complete, you can test this within your VR headset and then tweak the values further.

As your second challenge, set up hand poses for the other three interactable game objects. You simply need to replicate the steps you took to set up the M48. When setting up the hand pose for grabbing the magazines, you don’t need to curl the thumb as it will clip into the magazine. Also, you want to curl in all the remaining fingers. That is index, middle, ring and pinky fingers together as soon as the grip button on the controller is pressed.

You will need to set up each of these four fingers individually as their override values will differ. You can start off by selecting all four fingers and then set their input source property to override from the dropdown. Also, with all four fingers still selected, for their Apply Pose Values property, select the item’s Input Source and Override Value from the dropdown.

Now you need to select each of these four fingers individually and set their Override Value. Once you have achieved this for the right hand, replicate the same steps for setting up the grab pose for the left hand. Lastly, don’t forget to set up the grabbed and ungrabbed events on the interactable for which you are creating the hand pose.

The final completed version of the project for volume one will have all these hand poses set up for you should you want to explore the values I have used. Finally, let’s set up a unity layer to which all interactable objects within your scene will be assigned. You will call this layer Interactable. All interactable objects within your project must be assigned to this layer.

Navigate to the main menu and select Edit Project Settings to bring up the Project Settings window. From the pane on the left, select Tags and Layers. Within the right pane, expand the Layers dropdown. Ideally, user layer six should be the first unutilized layer within your project.

Within its text box, type in the value interactable. Now close the Project Settings window. In the Hierarchy, select all your interactable objects, namely the M48, the handgun, the M4 magazine, and the M1911 magazine.

In the Inspector, expand the layer dropdown and select the item Interactable. When prompted with the Change Layer dialog asking if you would like to set the layer to interactable for all child objects as well, select the button no, this object only. All your interactable objects have now been assigned to the interactable layer.

Now for this newly created interactable layer, you need to set up the Collision Matrix, which determines whether game objects assigned to different layers or even the same layer can collide with each other. From the main menu, select Edit Project Settings to open up the Project Settings window. From the pane on the left, select Physics.

Within the right pane, scroll down to the bottom and you will notice the Layer Collision Matrix with your newly added interactable layer displayed. Uncheck the checkbox located at the intersection between the interactable layers. This ensures that your interactables will not have physics interactions with each other. You may want to keep this checked in the event you believe that your interactables need to be able to interact with each other. Well, that’s all for this lecture, folks, and I will see you in the next one.