This article introduces project setup methods and SDK installation procedures for developing Meta Quest-compatible applications in Unity. By completing the steps in this article, you will have established the foundation for VR/AR application development using the Meta XR SDK.

1. Creating a Unity Project

[Creating a New Project]

• Open Unity Hub and click New project
• Select the Unity version in the New project window
  This article uses 6000.0.34f Install Android development environment when installing Unity
• Select Universal 3D from the templates
• Set the Project name in Project Settings
  This article sets it as *MetaXRSDK_Samples
• Click Create project to create the project

[Switching to Android Platform]

• Click File -> Build Profiles
• Select Android in the Build Profiles window and click Switch Platform

• Click Scene List in the left column
• Right-click and delete Scenes/SampleScene from the Scene List
• Close the Build Settings window

2. Installing Meta XR SDK

[Registering to My Assets on Asset Store]

• Open the Meta XR SDK page on Asset Store and add Meta XR All-in-One SDK to My Assets
  *Sign in with your account on the Unity page beforehand

[Installation via Package Manager]

• Return to Unity Editor, click Window -> Package Manager to open Package Manager
• Click My Assets and enter Meta in the search window

• Select Meta XR All in One SDK and click Install
  If using this asset for the first time, first click *Download. Install after SDK download is complete.
• After installation, you'll be prompted to restart the Editor, so click Restart Editor

• After restart, the Interaction SDK OpenXR Upgrade window may appear, but ignore and close it for now

3. Installing OpenXR Plugin

• After Unity Editor restart, open Package Manager again
• Click Unity Registry and search for OpenXR
• Install OpenXR Plugin
• A dialog prompting Meta XR Feature Set configuration will appear, so click Yes

• The Project Settings window will appear but can be ignored for now

4. Installing Unity OpenXR Meta (Optional)

The Unity OpenXR plugin installed in the above steps does not support Depth API occlusion features. If you plan to use Depth API, install Unity OpenXR Meta with the following steps.

• Open Package Manager
• Click Unity Registry and search for OpenXR
• Install Unity OpenXR Meta

5. Project Configuration

[Basic Configuration]

• Close the project once and reopen it just to be safe
• If the Interaction SDK OpenXR Upgrade dialog appears, click Use OpenXR Hand

[XR Plugin Management Configuration]

• Open Edit -> Project Settings
• Click XR Plugin Management in the Project Settings window
• Select the Android tab and turn OpenXR ON (others should be OFF)
• If using Meta Quest Link (PC VR for Windows), also turn OpenXR ON from the Windows tab

[Running Project Validation]

• Click Project Validation directly under XR Plugin Management
• Open the Android tab and click Fix All displayed in the tab information
  *Results may not be reflected immediately after clicking Fix All. In such cases, try opening other items and returning to Project Validation to check again
• If using Meta Quest Link, perform the same operation from the Windows tab

[Meta XR Configuration Verification]

• Click Meta XR
• Open the Android tab and click Fix All next to Outstanding Issues
• If using Meta Quest Link, perform the same operation from the Windows tab

[Creating AndroidManifest.xml]

• Close the Project Settings window
• In Unity Editor menu bar, click Meta -> Tools
• Click Create store-compatible AndroidManifest.xml
• If prompted for overwrite permission, click Replace

6. Build Settings (Optional)

This operation sets a custom app name (package name). While not mandatory, it's recommended when developing multiple apps.

[App Information Configuration]

• Open Edit -> Project Settings
• Select Player
• Set Company Name and Product Name

[Package Name Configuration]

• Select Android from the tabs displayed slightly below
• Open Other Settings
• Turn OFF the Override Default Package Name checkbox in Identification

7. Next Steps

The basic project configuration is now complete. Next, we'll explain how to display 3D objects in VR.

Displaying Objects in MetaQuest

8. Complete List of Meta XR SDK Articles

Getting Started with Meta XR SDK Quest App Development

This article introduces the steps to display 3D objects (cubes) in VR with Meta Quest. You'll learn the fundamentals of VR applications, from basic camera setup with Meta XR SDK to object placement and testing on actual hardware.
You can also check the operation in the 01-ShowObject scene of the sample published on GitHub.

This article continues from the following. Please check if your settings match.

Meta XR SDK Installation and Project Setup

1. Creating a Scene

[Creating a New Scene]

• Click File -> New Scene
• Select Basic(URP) in the New Scene window and click Create

• Click File -> Save
• Set the scene name and save
  This article uses *ShowObject

2. Setting Up the Quest Camera

[Placing the Camera Rig]

• Delete the Main Camera in the Hierarchy
• Enter CameraRig in the Project tab search area
• Select All from Search: directly under the search area
• Drag and drop OVRCameraRig into the Hierarchy
• If Project Setup Tool Fix Available appears, click Fix

[OVR Manager Configuration]

• Click OVRCameraRig in the Hierarchy
• Find OVR Manager in the Inspector to set detailed information for Quest
• Select Floor Level for Tracking Origin Type in the Tracking item
  *Tracking Origin Type is important for determining the origin height

Select Floor Level if you want to fix object display height relative to the room, and select *Eye Level if you want to fix objects relative to Quest's initial position.

3. Displaying Controllers and Hands

This article doesn't introduce interactions with hands or controllers, but these will be used in subsequent articles. Also, it's enjoyable to visually confirm that hands and controllers are recognized by Quest during testing, so I'll introduce the setup procedure.

[Adding Interaction Features]

• Enter OVRInteractionComprehensive in the Project tab search area
• Select All from Search: directly under the search area
• Drag and drop OVRInteractionComprehensive into OVRCameraRig as a child element

[Hand Tracking Configuration]

• Click OVRCameraRig in the Hierarchy
• Find OVR Manager in the Inspector to set detailed information for Quest
• Find Hand Tracking Support in the Quest Features section and select your preferred mode
  This series assumes *Controllers and Hands is selected.

　 [Always Display Controller Ray (Optional)]

The Ray object that indicates the controller's pointing direction is not always displayed. Of course, this is fine as is, but if you want the Ray object to always be displayed, you can set it up with the following procedure.

• Open LeftInteractions child element of OVRInteractionComprehensive
• Further open Interactors -> Controller -> ControllerRayInteractor -> Visuals
• Click ControllerRay child element of Visuals
• Pay attention to Controller Ray Visual in the Inspector
• Turn OFF the Hide When No Interactable checkbox
• Perform the same operation for RightInteractions to set the right hand Ray

4. Object Placement

[Creating a Cube]

• Right-click in the empty space of the Hierarchy with no other objects selected
• Click 3D Object -> Cube

[Position and Size Adjustment]

• Click the Cube added to the Hierarchy
• Set the Cube's position, rotation, and size in the Transform section of the Inspector
  *This article uses the following settings to display a slightly tilted 0.3m cube at 1.3m height from the floor, 1m ahead from the origin

5. Testing

[Installing on Actual Device]

• Connect Quest and PC with USB cable
• Click File -> Build Settings in Unity Editor
• Click Build And Run
• Set installer (apk) name in alphanumeric characters and save
• After installer generation and installation complete, the app automatically starts in Quest
• A cube should appear in front of you. If it appears in a different direction, press and hold the meta (oculus) button on the right controller to reset orientation
• If installation errors occur, review settings or create only the apk with Build and install the app from SideQuest or MetaQuestDeveloperHub

[Using Meta Quest Link (Windows)]

For details, see the official page.

• Connect Quest and PC with USB cable
• Start Quest Link in Quest
• Click the Play button in Unity Editor
• A cube should appear in front of you. If it appears in a different direction, press and hold the meta (oculus) button on the right controller to reset orientation

6. Next Steps

You've now achieved basic object display in VR. Next, we'll explain AR display using the passthrough feature.

AR Display Using MetaQuest Passthrough

7. Complete List of Meta XR SDK Articles

Getting Started with Meta XR SDK Quest App Development

This article introduces the steps to display 3D objects (cubes) in AR with Meta Quest. You'll learn basic techniques for overlaying virtual objects on the real world, from passthrough feature setup to background transparency.
You can also check the operation in the 01-ShowObject-AR scene of the sample published on GitHub.

This article edits the scene created in the following article to make it AR. Please read this first. However, if you already have a VR scene using Meta XR SDK, you don't need to check the article below.

Displaying Objects in MetaQuest

1. Duplicating the Scene

While it's possible to edit the scene created in the previous article, to try AR conversion without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open the scene you want to convert to AR
  *This article uses the VR scene ShowObject created previously
• Click File -> Save As... to save the current scene with a new name
  This article uses *ShowObject-AR
• Confirm that the scene name displayed in Hierarchy is ShowObject-AR

2. Enabling Passthrough Feature

[Passthrough Settings in OVR Manager]

• Select OVRCameraRig in the Hierarchy
• Find OVR Manager among the detailed information displayed in the Inspector
• Change Passthrough Support in the Quest Features section to Supported or Required
• Turn ON the Enable Passthrough checkbox in the Insight Passthrough & Guardian Boundary section

[Adding Passthrough Layer]

The operations up to this point enable the passthrough function itself, but passthrough video won't be displayed yet, so add a layer to display the video with the following steps.

• Click Add Component at the bottom of OVRCameraRig's Inspector
• Search for Passthrough in the search area
• Click OVRPassthroughLayer
• Change Placement to Underlay in the OVRPassthroughLayer added to the Inspector

3. Camera Background Settings

The passthrough display setup is mostly complete with the above steps, but if you build now, only the Skybox will be displayed and AR won't work. Finally, change the camera background rendering settings to enable passthrough video display.

[CenterEyeAnchor Settings]

• Open the TrackingSpace child element of OVRCameraRig in the Hierarchy
• Select CenterEyeAnchor child element of TrackingSpace

[Background Transparency]

• Find Camera in the Inspector and change Background Type from Skybox to Solid Color
• Change the color in the Background section to all RGBA values to 0
  Setting *A (alpha value) to 0 for transparency is especially important

4. Testing

[Installing on Actual Device]

• Connect Quest and PC with USB cable
• Click File -> Build Settings in Unity Editor
• Click Build And Run
• Set installer (apk) name in alphanumeric characters and save
• After installer generation and installation complete, the app automatically starts in Quest
• The cube should be overlaid on the real world. If it appears in a different direction, press and hold the meta (oculus) button on the right controller to reset orientation
• If installation errors occur, review settings or create only the apk with Build and install the app from SideQuest or MetaQuestDeveloperHub

[Using Meta Quest Link (Windows)]

For details, see the official page.

• Connect Quest and PC with USB cable
• Start Quest Link in Quest
• Click the Play button in Unity Editor
• The cube should be overlaid on the real world. If it appears in a different direction, press and hold the meta (oculus) button on the right controller to reset orientation

5. Next Steps

You've now achieved the basics of AR display. However, controller warp (teleport) and turn (viewpoint rotation) are still enabled. While these features are useful in VR, they may compromise consistency with real space in AR.

To achieve a more natural AR experience, next we'll explain how to disable these locomotion features.

Disabling MetaQuest Locomotion

6. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

The OVRInteractionComprehensive used in this series includes objects with basic settings for hand and controller interactions already configured. Among these features, locomotion functions for teleportation and viewpoint rotation using joysticks are also provided and are enabled by default.

However, depending on the content (especially AR content), these features may be unnecessary. This article introduces how to disable unnecessary locomotion features.

This explanation assumes that basic settings for VR and AR implementation are already complete. If you're interested in basic settings for VR or AR development, please also read the following articles.

[VR Version]

Displaying Objects in MetaQuest

[AR Version]

AR Display Using MetaQuest Passthrough

1. Disabling Locomotion Features

[Disabling Right Hand Controller Locomotion]

• Find OVRCameraRig in the Hierarchy
• Open the OVRInteractionComprehensive child element of OVRCameraRig
• Further open child elements in this order: RightInteractions -> Interactors -> Controller -> LocomotionControllerInteractorGroup
• Deactivate each of the following child elements of LocomotionControllerInteractorGroup:
  • TeleportControllerInteractor (Teleportation feature)
  • ControllerTurnerInteractor (Viewpoint rotation feature)
  • ControllerSlideInteractor

[Disabling Left Hand Controller Locomotion]

• Perform the same settings for LeftInteractions
• Open child elements in this order: LeftInteractions -> Interactors -> Controller -> LocomotionControllerInteractorGroup
• Deactivate each of the following child elements of LocomotionControllerInteractorGroup:
  • TeleportControllerInteractor
  • ControllerTurnerInteractor
  • ControllerSlideInteractor

[Disabling Locomotor Object]

• Deactivate the Locomotor child element of OVRInteractionComprehensive

2. Features That Will Be Disabled

With the above settings, the following locomotion features will be disabled:

Disabling these features is especially recommended for AR applications to maintain consistency with real space.

3. Next Steps

You've now completed disabling locomotion features. Next, we'll explain preparation for object manipulation features.

Grabbing Objects in MetaQuest (Preparation)

4. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

We begin introducing how to grab and manipulate 3D objects (cubes) with Meta Quest. While "grabbing objects" in VR or AR may sound simple, the required settings differ depending on whether the object is nearby or distant.

This series will introduce implementation methods for each case, but as a preparatory stage, this article explains settings commonly required for all cases. Completing this article will establish the foundation for object manipulation.

This video is the AR version but supports both AR/VR The content of this article is preparatory, so manipulation like in the video cannot be performed with this content alone

This article assumes manipulation of cubes displayed in VR or AR scenes created in the following articles. Starting from cubes without special components added, in a state where they can only be viewed with Quest, you can achieve object manipulation by implementing the content of this article and subsequent articles if you have a similar situation in other projects.

[VR Version]

Displaying Objects in MetaQuest

[AR Version]

AR Display Using MetaQuest Passthrough

1. Duplicating the Scene

While it's possible to edit the scene created in the previous article, to try manipulation without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open the VR or AR scene created in previous articles
• Click File -> Save As... to save the current scene with a new name
  This article uses *Manipulation
• Confirm that the scene name displayed in Hierarchy is Manipulation

2. Enabling Object Contact Detection

To grab and manipulate objects, you need to detect contact between hands/controllers and objects. The procedure for object contact detection explained here is common to Unity development, not limited to Quest development.

[Checking Collider]

• Select the target object (Cube) in the Hierarchy
• Confirm that BoxCollider is applied in the Cube details displayed in the Inspector
  For objects other than Cube, use Colliders appropriate for each object For custom models, manually add Colliders

[Adding and Configuring Rigidbody]

• Click the Add Component button at the bottom of the Inspector
• Enter Rigidbody in the search area
• Select Rigidbody from the displayed candidates
• Confirm that Rigidbody has been added to the Inspector and change it as follows:

Important: Collider and Rigidbody are necessary for contact detection, so be sure to set them on target objects.

3. Adding Grabbable and Movable Attributes

Here we add two types of attributes: making it grabbable and making it movable.

[Adding Grabbable Component]

• Select Cube in the Hierarchy
• Click Add Component, then search for Grabbable
• Select Grabbable from the candidates

[Adding Grab Free Transformer]

• Click Add Component in Cube's Inspector
• Search for Grab Free
• Select Grab Free Transformer from the candidates
  Script that enables movement, rotation, and scale changes when grabbed by hands or controllers Can set constraints on movement and rotation through detailed configuration
  *Other options like One Grab Rotate Transformer that restricts only rotation are also available - search with the Transformer keyword if interested

[Single-Hand Operation Setup]

• Open Options in Grabbable
• Drag and drop Cube to the right area of One Grab Transformer
  *This operation registers the Grab Transformer added to Cube with the Grabbable script, completing preparation for moving objects with one hand

[Two-Hand Operation Setup]

Skip this if only single-hand operation is needed

• Open Options in Grabbable again
• Drag and drop Cube to the right area of Two Grab Transformer
  *This operation registers the Grab Transformer added to Cube with the Grabbable script, completing preparation for moving objects with both hands
• The Grabbable options should look like below if configured correctly

[Scale Change Settings (Optional)]

If you want to change object scale when operating with both hands, also perform the following settings.

• Pay attention to Grab Free Transformer in Cube's Inspector
• Open Scale Constraints
• Uncheck all X, Y, Z Axis Constraint checkboxes

4. Next Steps

You've now completed settings for changing the object's (Cube's) position, rotation, and scale when grabbed by controllers or hands.

However, you still can't manipulate the object. As explained at the beginning, this is because the required settings differ depending on the distance between objects and controllers/hands, and we haven't configured these yet. Additionally, we haven't set whether to use controllers, hands, or both for grabbing.

The next articles will introduce configuration methods for both near and far cases.

[Nearby Objects Edition]

Grabbing Nearby Objects in MetaQuest

[Distant Objects Edition]

Grabbing Distant Objects in MetaQuest

5. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

This article introduces how to grab and manipulate nearby 3D objects (cubes) with Meta Quest. This video is the AR version but supports both AR/VR.
You can also check the operation in the 02-NearManipulation or 02-NearManipulation-AR scene of the sample published on GitHub.

This article assumes that the preliminary preparations in the following article have been completed. Please note this.

[Preparation Edition]

Grabbing Objects in MetaQuest (Preparation)

1. Duplicating the Scene

While it's possible to edit the scene created in the previous article, to try manipulation without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open the Manipulation VR or AR scene created in previous articles
• Click File -> Save As... to save the current scene with a new name
  This article uses *NearManipulation
• Confirm that the scene name displayed in Hierarchy is NearManipulation

2. Settings for Grabbing Nearby Objects with Controllers or Hands

I'll introduce the scripts to add and their configuration methods for cases when using controllers or hands to grab objects. By configuring both, you can use both controllers and hands.

[Grabbing with Controllers]

• Display Cube's Inspector and click Add Component
• Search for Grab Interactable and select Grab Interactable from candidates
• Drag and drop Cube to Pointable Element in the added Grab Interactable
  *Connects Grab Interactable and Grabbable, reflecting controller grab information to the behavior set in Grabbable (position, rotation, scale changes)
• Further drag and drop Cube to Rigidbody in Grab Interactable
  *By assigning the Rigidbody added to Cube in the previous article, contact detection between controller and Cube is utilized

[Grabbing with Hands]

• Display Cube's Inspector and click Add Component
• Search for Grab Interactable and select Hand Grab Interactable from candidates
• Drag and drop Cube to Pointable Element in the added Hand Grab Interactable
  *Connects Hand Grab Interactable and Grabbable, reflecting hand grab information to the behavior set in Grabbable (position, rotation, scale changes)
• Further drag and drop Cube to Rigidbody in Hand Grab Interactable
  *By assigning the Rigidbody added to Cube in the previous article, contact detection between hand and Cube is utilized

[Note!]
If ControllerOnly is selected in Hand Tracking Support of OVR Manager attached to the OVRCameraRig object, hand tracking won't work and you won't be able to grab objects with hands. Please check the settings.

3. Fine-tuning Controller Settings (Optional)

The above operations enable grabbing and moving objects with controllers. By default, the grip button (middle finger) is used for grabbing. Here's how to enable grabbing with the trigger button as well.

• Open the OVRInteractionComprehensive child element of OVRCameraRig in the Hierarchy
• Further open child elements in this order: RightInteractions -> Interactors -> Controller and No Hand -> ControllerGrabInteractor
• Open the GripButtonSelector child element of ControllerGrabInteractor
• Pay attention to ControllerSelector in GripButtonSelector's Inspector
• Turn ON the TriggerButton checkbox in Controller Button Usage
  *Turn off the GripButton checkbox if GripButton is not needed
• Perform the same operation for the left hand controller (LeftInteractions) to set the button used for grabbing

4. Fine-tuning Hand Behavior (Optional)

This is a detailed point, but by default when scaling objects with both hands, the hand objects don't follow the actual hands. If you want the hand objects to follow, try the following.

• Select Cube in the Hierarchy
• Pay attention to Hand Grab Interactable registered to Cube in the Inspector
• Change Hand Alignment setting to None

6. Next Steps

You've now achieved grabbing and manipulating nearby objects. You can also combine near and far object manipulation for the same object.

If you want to add distant object manipulation, follow the procedures from chapter 2 of the next article.

Grabbing Distant Objects in MetaQuest

7. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

This article introduces how to grab and manipulate distant 3D objects (cubes) with Meta Quest. This video is the AR version but supports both AR/VR.
You can also check the operation in the 03-FarManipulation or 03-FarManipulation-AR scene of the sample published on GitHub.

This article assumes that the preliminary preparations in the following article have been completed. Please note this.

[Preparation Edition]

Grabbing Objects in MetaQuest (Preparation)

1. Duplicating the Scene

While it's possible to edit the scene created in the previous article, to try manipulation without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open the Manipulation or NearManipulation VR or AR scene created in previous articles
• Click File -> Save As... to save the current scene with a new name
  This article uses *FarManipulation
• Confirm that the scene name displayed in Hierarchy is FarManipulation

2. Settings for Grabbing Distant Objects with Controllers or Hands

I'll introduce the scripts to add and their configuration methods for cases when using controllers or hands to grab objects. By configuring both, you can use both controllers and hands.

[Grabbing with Controllers]

• Display Cube's Inspector and click Add Component
• Search for Grab Interactable and select Distance Grab Interactable from candidates
• Drag and drop Cube to Pointable Element in the added Distance Grab Interactable
  *Connects Distance Grab Interactable and Grabbable, reflecting controller grab information to the behavior set in Grabbable (position, rotation, scale changes)
• Further drag and drop Cube to Rigidbody in Distance Grab Interactable
  *By assigning the Rigidbody added to Cube in the previous article, contact detection between controller pointer and Cube is utilized

[Grabbing with Hands]

• Display Cube's Inspector and click Add Component
• Search for Grab Interactable and select Distance Hand Grab Interactable from candidates
• Drag and drop Cube to Pointable Element in the added Distance Hand Grab Interactable
  *Connects Distance Hand Grab Interactable and Grabbable, reflecting hand grab information to the behavior set in Grabbable (position, rotation, scale changes)
• Further drag and drop Cube to Rigidbody in Distance Hand Grab Interactable
  *By assigning the Rigidbody added to Cube in the previous article, contact detection between hand pointer and Cube is utilized

[Note!]

If ControllerOnly is selected in Hand Tracking Support of OVR Manager attached to the OVRCameraRig object, hand tracking won't work and you won't be able to grab objects with hands. Please check the settings.

3. Current Status Check and Behavior Modification

The following video shows the behavior of content created with the operations so far. (For controller grabbing, use the middle finger grip button.)

As you can see, the default Meta XR SDK setting is to pull distant objects towards you, which differs from the behavior shown in the opening video.
While this may be fine for some content, there are cases where you want to change only the position without pulling objects towards you, so I'll introduce how to achieve this.

[Object Behavior When Grabbed]

• Select Cube and click Add Component at the bottom of the Inspector
• Search for Move From and select Move From Target Provider to add

About Behavior Differences:

• Default: Objects move to controller or hand position before manipulation begins
• After Change: Objects can be manipulated while maintaining their original position

Use Move From Target Provider with the following steps to set objects to maintain their original position.

[Behavior Adjustment for Controller Use]

• Pay attention to Distance Grab Interactable in the Inspector
• Open Optionals
• Drag and drop Cube to Movement Provider to associate the Move From Target Provider registered to this Cube

[Behavior Adjustment for Hand Use]

• Pay attention to Distance Hand Grab Interactable in the Inspector
• Turn Reset Grab On Grabs Updated to OFF
• Open Optionals
• Drag and drop Cube to Movement Provider to associate the Move From Target Provider registered to this Cube

4. Fine-tuning Controller Settings (Optional)

The above operations enable grabbing and moving objects with controllers. By default, the grip button (middle finger) is used for grabbing. Here's how to enable grabbing with the trigger button as well.

• Open the OVRInteractionComprehensive child element of OVRCameraRig in the Hierarchy
• Further open child elements in this order: RightInteractions -> Interactors -> Controller and No Hand -> ControllerDistanceGrabInteractor
• Open the GripButtonSelector child element of ControllerDistanceGrabInteractor
• Pay attention to ControllerSelector in GripButtonSelector's Inspector
• Turn ON the TriggerButton checkbox in Controller Button Usage
  *Turn off the GripButton checkbox if GripButton is not needed
• Perform the same operation for the left hand controller (LeftInteractions) to set the button used for grabbing

5. Fine-tuning Hand Behavior (Optional)

This is a detailed point, but by default when scaling Cube with both hands, the hand objects fly towards the Cube. If you want the hand objects to follow the actual hands, try the following.

• Select Cube in the Hierarchy
• Pay attention to Distance Hand Grab Interactable registered to Cube in the Inspector
• Change Hand Alignment setting to None

6. Next Steps

You've now achieved grabbing and manipulating distant objects. You can also combine distant and nearby object manipulation for the same object.

If you want to add nearby object manipulation, follow the procedures from chapter 2 of the next article.

Grabbing Nearby Objects in MetaQuest

7. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

This time we'll cover interaction with pointers (Rays) emitted from controllers or hands in Meta Quest. Specifically, we'll introduce procedures to implement Hover (pointing) and selection (click) detection using Rays.
You can also check the operation in the 04-RayInteraction or 04-RayInteraction-AR scene of the sample published on GitHub.

*This video is the AR version but supports both AR/VR.

This article assumes manipulation of cubes displayed in VR or AR scenes created in the following articles. Starting from cubes without special components added, in a state where they can only be viewed with Quest, you can achieve object manipulation by implementing the content of this article and subsequent articles if you have a similar situation in other projects.

[VR Version]

Displaying Objects in MetaQuest

[AR Version]

AR Display Using MetaQuest Passthrough

1. Duplicating the Scene

While it's possible to edit the scene created in the above articles, this chapter introduces how to duplicate the scene to try Ray interaction without destroying the existing scene. Skip this if unnecessary.

• Open the VR or AR scene created in the above articles
• Click File -> Save As... to save the current scene with a new name
  *This article uses RayInteraction
• Confirm that the scene name displayed in Hierarchy is RayInteraction

2. Enabling Ray-Object Intersection Detection

To implement interaction with Rays, add a script to the object that detects interaction with Rays.

• Select the target object (Cube) in the Hierarchy
• Confirm that BoxCollider is applied in the Cube details displayed in the Inspector
  For objects other than Cube, set Colliders appropriate for each object For custom models, manually add Colliders
• Click the Add Component button at the bottom of the Inspector
• Enter Interactable in the search area
• Select Ray Interactable from the displayed candidates

When you check the added Ray Interactable, the Surface item shows None. This item is required as it registers the shape for determining Ray intersection with Ray Interactable.

• Click Add Component at the bottom of Cube's Inspector
• Search for Collider and select Collider Surface
• Drag and drop Cube to Collider Surface
  *Uses Cube's collider shape as the shape for intersection detection
• Drag and drop Cube to Surface in RayInteractable
  *Registers the shape information (Surface) created above to RayInteractable

3. Getting Ray Interaction Events

With the above operations, you're ready for Ray interaction, so add functionality to receive Hover (pointing) and selection (click) as events.

• Select Cube in the Hierarchy
• Search for Interactable in Add Component at the bottom of Cube's Inspector
• Select Interactable Unity Event Wrapper
• Drag and drop Cube to Interactable View in Interactable Unity Event Wrapper
  *Registers Ray Interactable to connect to Hover and Select event firing

4. Testing Ray Interaction Operation

Actually receive events and test operation. This article displays Hover and Select states as text.

• Right-click in the empty space of the Hierarchy
• Click 3D Object -> Text - TextMeshPro
  *If TextMeshPro is not installed, install TextMeshPro following the dialog instructions. (Sample installation is unnecessary)
• Change Text(TMP) settings in the Inspector as follows to display text above the Cube

• Pay attention to TextMeshPro-Text in the Inspector
• Change text alignment to center in Alignment
• Click Cube and pay attention to Interactable Unity Event Wrapper in the Inspector
• Click + at the bottom right of WhenHover
• Drag and drop Text(TMP) to the area marked None
• Open the dropdown menu marked NoFunction
• Click TextMeshPro -> string text
• Enter the text you want to display when Hover starts in the text area below the dropdown menu (example: Hover)

• The above operation will display Hover when Hover (pointing) starts. Set text to display for the following events in the same way
  When Unhover() : Un-Hover
  When Select() : Select
  When Unselect() : Un-Select

When testing operation, Hover or Un-Hover is displayed when Ray intersection starts and ends as in the opening video, and Select, Un-Select is displayed when finger pinch or controller trigger press starts and ends.

5. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

This article introduces button interaction as an example of operating Unity UI with Meta Quest. First, we'll confirm that hands and controllers can interact with buttons, then explain how to make a Cube disappear when a button is pressed as an example of interaction.
You can also check the operation in the 05-UiInteraction or 05-UiInteraction-AR scene of the sample published on GitHub.

*This video is the AR version but supports both AR/VR.

This article assumes manipulation of cubes displayed in VR or AR scenes created in the following articles. Starting from cubes without special components added, in a state where they can only be viewed with Quest, you can achieve object manipulation by implementing the content of this article and subsequent articles if you have a similar situation in other projects.

[VR Version]

Displaying Objects in MetaQuest

[AR Version]

AR Display Using MetaQuest Passthrough

1. Duplicating the Scene

While it's possible to edit the scene created in the above articles, to try UI operation without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open the VR or AR scene created above
• Click File -> Save As... to save the current scene with a new name
  This article uses *UiInteraction
• Confirm that the scene name displayed in Hierarchy is UiInteraction

2. Creating UI

Create a button and place it in 3D space.

[Creating a Button]

• Right-click in the empty space of the Hierarchy
• Click UI -> Button - TextMeshPro
• If TMP Importer (Text Mesh Pro installer) appears, click Import TMP Essentials
• Close TMP Importer
• Confirm that Canvas and EventSystem have been added

[3D Space Placement Settings]

• Select Canvas and find the Canvas component in the Inspector
• Change Canvas Render Mode from World Space
  *By default, UI is placed on screen, but this change places it in 3D space
• Change Button child element PosX and PosY to 0
• Operate Canvas Inspector parameters to adjust UI size as follows
  Position X: 0 Y: 1.2 Z: 0.8
  Size Width: 200 Height: 90
  Scale X: 0.003 Y: 0.003 Z: 0.003
  *The white frame in the figure represents the Canvas area

3. Making Canvas Quest-Compatible

At this stage, we've only created UI and can't interact with controllers or hands used in Meta Quest, so we'll configure basic interaction settings.

[Pointable Canvas Settings]

• Click Canvas and click Add Component at the bottom of the Inspector
• Search for Pointable and click Pointable Canvas from the displayed candidates to add
• Drag and drop the Canvas object in Hierarchy to the right area of Canvas in the added Pointable Canvas

[EventSystem Settings]

• Select EventSystem in Hierarchy
• Click Add Component and search for Pointable Canvas
• Click Pointable Canvas Module to add
  *Uses Pointable Canvas input provided by Meta instead of normal input
• Turn ON the Exclusive Mode checkbox in Pointable Canvas Module

4. UI Detection Area Settings in 3D Space

With Meta XR SDK, you need to separately set detection areas in 3D space for interaction between UI (Canvas) and hands or controllers.

[Creating Contact Detection Object]

• Right-click in the empty space of Hierarchy and click Create Empty
• Name the generated GameObject Surface
  *Any name is fine
• Click Surface and pay attention to Transform
• Set Surface object position and size same as Canvas
  Position X: 0 Y: 1.2 Z: 0.8
  Scale X: 0.6 Y: 0.27 Z: 1
  Scale is calculated as Canvas Width × Scale(x), Height × Scale(Y). Z can be anything It's okay to make it slightly larger or smaller than Canvas

[Adding Surface-Related Components]

• Click Add Component in Surface's Inspector

• Search for Plane Surface and click Plane Surface to add
  *This prepares the plane at Surface position (= Canvas position) for pointer contact

• Click Add Component in Surface's Inspector

• Click Clipped Plane Surface from candidates to add
  *Pointer contact detection will only occur in the clipping area

• Click Add Component in Surface's Inspector

• Search for Bounds and click Bounds Clipper to add
  *Script to set clipping area is added

[Associating Components]

• Pay attention to Plane Surface in Clipped Plane Surface
• Drag and drop Surface object
  *Applies clipping to Surface object's plane
• Open Clippers in Clipped Plane Surface and click + button
• Drag and drop Surface object to Element 0
  *Clips with the area set by Bounds Clipper

[Optional]

• Finally, drag and drop Surface object to Canvas to make it a child element
• Confirm that the child Surface Position is 0 0 0 just to be safe

5. Selecting Interaction Method

Set whether UI responds to Rays from controllers or hands, or Poke operations touching UI directly nearby. You can set either one or both.

[Remote Operation Using Ray]

• Click Canvas and click Add Component at the bottom of the Inspector
• Search for Ray Interactable and add Ray Interactable from displayed candidates
• Drag and drop Canvas object to Pointable Element in the added Ray Interactable
  *Links Ray operation and Pointable Canvas to enable UI operation within Canvas
• Drag and drop Surface object to Surface in Ray Interactable
• If a dialog appears, select ClippedPlaneSurface
  *Enables interaction only in the area set by Clipped Plane Surface

[Proximity Operation Using Poke]

• Click Canvas and click Add Component at the bottom of the Inspector
• Search for Poke Interactable and add Poke Interactable from displayed candidates
• Drag and drop Canvas object to Pointable Element in the added Poke Interactable
  *Links Poke operation and Pointable Canvas to enable UI operation within Canvas
• Drag and drop Surface object to Surface Patch in Poke Interactable
• If a dialog appears, select ClippedPlaneSurface
  *Enables interaction only in the area set by Clipped Plane Surface

[Basic Interaction Confirmation]

With the above operations, interaction with buttons becomes possible. When testing operation, you can confirm that the button color changes slightly when clicking the button as in the following video.

6. Button Event Settings

The settings here are not specific to Meta XR SDK but relate to general Unity UI behavior. As an example, let's make the Cube disappear when a button is pressed.

• Click Button and find On Click in the Inspector
• Click the + button at the bottom right
• Drag and drop Cube to the area marked None in the newly added area
• Open the dropdown menu marked No Function
• Click GameObject -> SetActive
• Confirm that the checkbox is OFF
• This will make Cube inactive when the button is pressed

7. Testing Operation

[Installing on Actual Device]

• Connect Quest and PC with USB cable
• Click File -> Build Settings in Unity Editor
• Click Build And Run
• Set installer (apk) name in alphanumeric characters and save

[Using Meta Quest Link (Windows)]

• Connect Quest and PC with USB cable
• Start Quest Link in Quest
• Click the Play button in Unity Editor

You can confirm that the Cube disappears when clicking the button with either Ray or Poke.

8. Summary of Quest UI Interaction Setup Key Points

Since the operation procedures were numerous, here's a summary of key points required for Quest UI interaction.

[Making Canvas Quest-Compatible]

• Attach Pointable Canvas to Canvas

[Making EventSystem Quest-Compatible]

• Attach Pointable Canvas Module to EventSystem

[Define UI Position and Range for Quest Interaction]

• Define plane with Plane Surface (infinite plane)
• Limit range with Clipped Plane Surface and Bounds Clipper

[Specify Interaction Method]

• Ray Operation: Add Ray Interactable to Canvas
• Poke Operation: Add Poke Interactable to Canvas

With the above settings, Unity UI interaction with Meta Quest becomes possible.

9. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

While previous articles introduced basic hand manipulation implementation methods, actual applications require more natural and intuitive operation feeling. This article explains manipulation techniques that use Hand Pose to record the relative positional relationship between objects and hands, faithfully reproducing how things are held in the real world. This feature makes it possible to operate objects like tools or instruments where specific holding methods are important with a more realistic feeling.
You can also check the operation in the 06-HandPoseManipulation or 06-HandPoseManipulation-AR scene of the sample published on GitHub.

This article assumes Windows environment development to use QuestLink.

*This video is the AR version but supports both AR/VR.

This article proceeds based on VR or AR scenes created in the following articles.

[VR Version]

Displaying Objects in MetaQuest

[AR Version]

AR Display Using MetaQuest Passthrough

1. Duplicating the Scene

While it's possible to edit the scene created in the previous article, to try Hand Pose manipulation without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open a VR or AR compatible scene
• Click File -> Save As... to save the current scene with a new name
  *This article uses HandPoseManipulation
• Confirm that the scene name displayed in Hierarchy is HandPoseManipulation

2. Object Preparation

We'll use a Cylinder object for Hand Pose manipulation demonstration.

• Delete Cube from Hierarchy
• Right-click in the empty space of Hierarchy with nothing selected
• Click 3D Object -> Cylinder

3. Creating Object Structure for Hand Pose Manipulation

To perform manipulation following hands, we use Grabbable and Hand Grab Pose RecorderGameObject introduced in previous articles, but the manipulation target object requires a slightly different structure setup. Specifically, create a parent GrabRoot object and place Cylinder as its child element.

• Right-click in the empty space of Hierarchy
• Click Create Empty
• Change the created GameObject name to GrabRoot

[GrabRoot Object Settings]

• Change GrabRoot object position and size as follows
  Position: 0 1.3 0.4
  Rotation: 0 0 0
  Scale: 1 1 1
  *Root Scale must be 1

[Cylinder Placement]

• Drag Cylinder to move it as a child element of GrabRoot

• Change Cylinder object position and size as follows
  Position: 0 0 0
  Rotation: 0 0 0
  Scale: 0.03 0.1 0.03

4. Basic Manipulation Feature Settings

[Physics Behavior Settings]

• Click GrabRoot
• Click Add Component in the Inspector
• Search for Rigidbody and add
  *Used for contact detection as mentioned earlier
• Set RigidBody Use Gravity and Is Kinematic as follows
  Use Gravity: OFF
  Is Kinematic: ON

[Adding Manipulation Features]

• Search for Grabbable in GrabRoot's Inspector Add Component and add
  *Used to enable manipulation as mentioned earlier

5. Creating Hand Shape Data

Here we create information about hand shapes you want objects to follow and add manipulation features. We use tools provided by the SDK to create hand shape data.

[Starting Hand Grab Pose Recorder]

• Click Meta in the menu bar
• Click Interaction -> Hand Grab Pose Recorder in order
• Pay attention to 1 in the Hand Grab Pose Recorder dialog
• Select the hand you want to record shapes for (example: RightInteractions) in Hand used for recording poses
• Select GrabRoot in GameObject to record the hand grab poses for

6. Recording Hand Shapes

[Recording Preparation]

• Connect Quest and PC with Link cable
• Start Quest Link in Quest
• Click the Play button in Unity Editor

[Recording Hand Shapes]

• Press the Space key on keyboard while pretending to grab the Cylinder
• Hand shape is recorded

[Saving Recording Data]

• Return to Hand Grab Pose Recorder while in Play mode and click Save To Collection
• File with recorded hand shape is saved
• Stop Play mode
• Finally click Load From Collection
• HandGrabInteractable is automatically added to GrabRoot child elements

7. Manipulation Feature Adjustments

Hand manipulation is now possible with the above operations, but currently you can grab not only by grasping with hands but also by pinching with fingers. Also, manipulation only supports the right hand. So we'll add procedures to resolve these issues.

[Restricting Grab Methods]

• Click HandGrabInteractable child element of GrabRoot
• Find Supported Grab Types in the Inspector
• Change this content from All to Palm
  *This disables pinch manipulation and restricts to palm grasping only

[Adding Left Hand Support]

• Click Create Mirrored HandGrabInteractable at the bottom of the Inspector to also create left hand Interactable
  *This enables similar manipulation with the left hand

8. Testing Operation

[Using Meta Quest Link (Windows)]

For details, see the official page.

• Connect Quest and PC with USB cable
• Start Quest Link in Quest
• Click the Play button in Unity Editor
• Try grabbing the Cylinder with the recorded hand shape. You should be able to confirm that the object can be grabbed naturally according to the hand shape

[Installing on Actual Device]

• Connect Quest and PC with USB cable
• Click File -> Build Settings in Unity Editor
• Click Build And Run
• Set installer (apk) name in alphanumeric characters and save
• After installer generation and installation complete, the app automatically starts in Quest

9. Further Customization

Using Hand Grab Pose Recorder, you can record more complex hand shapes or multiple holding methods. For example:

• Recording multiple holding methods for the same object
• Recording shapes for different hand sizes
• Fine adjustments matching more natural hand movements

For these details, check the various setting options in Hand Grab Pose Recorder.

10. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index

This article introduces how to implement PalmMenu that displays UI on the palm and operates it with fingers of the other hand in Meta Quest. PalmMenu can achieve more natural interaction that appeals to the sense of touch compared to floating UI in mid-air, as you actually trace or tap the palm to operate it, making it a very useful feature.
You can also check the operation in the 07-PalmMenu or 07-PalmMenu-AR scene of the sample published on GitHub.

*This video is the AR version but supports both AR/VR.

This article proceeds based on VR or AR scenes created in the following articles.

[VR Version]

Displaying Objects in MetaQuest

[AR Version]

AR Display Using MetaQuest Passthrough

1. Duplicating the Scene

While it's possible to edit the scene created in the previous article, to try PalmMenu without destroying the existing scene, I'll introduce how to duplicate the scene. Skip this if unnecessary.

• Open a VR or AR compatible scene
• Click File -> Save As... to save the current scene with a new name
  *This article uses PalmMenu
• Confirm that the scene name displayed in Hierarchy is PalmMenu

2. Scene Preparation

• Delete the Cube used in the duplicate source as it's unnecessary
• Enter PalmMenu in the Project tab search area
• Select All from Search: directly under the search area
• Two PalmMenus will be found, but select the one stored in the following directory
  Packages/com.meta.xr.sdk.interaction.ovr/Runtime/Sample/Prefabs/
  *Folder name is displayed at the bottom of the Project tab

• Drag and drop PalmMenu prefab into Hierarchy
• Right-click in the empty space of Hierarchy
• Click UI -> EventSystem
• Confirm that EventSystem has been added to Hierarchy
• Select EventSystem and display Inspector
• Click Add Component at the bottom of Inspector
• Search for Pointable Canvas Module and add
• Right-click PalmMenu object
• To make editing easier later, click Prefab -> Unpack Completely

3. PalmMenu Settings

Hand Ref Settings

PalmMenu has two Hand Refs attached to link with hand tracking status, but Hand is unset with None. Associate these with left and right hand information.

[Associate first Hand Ref with left hand]

• Open OVRInteractionComprehensive child element of OVRCameraRig object
• Confirm that there's LeftInteractions as its child element

• Click PalmMenu
• Drag and drop LeftInteractions to Hand in the first Hand Ref

[Associate second Hand Ref with right hand]

• Open OVRInteractionComprehensive child element of OVRCameraRig object
• Confirm that there's RightInteractions as its child element
• Click PalmMenu
• Drag and drop RightInteractions to Hand in the second Hand Ref

Dominant Hand Game Object Filter Settings

Next, specify objects (fingertips) that can interact with UI in the Dominant Hand Game Object Filter attached to PalmMenu.

[Register left hand fingertip object]

• Open OVRInteractionComprehensive child element of OVRCameraRig object
• Further open child elements in this order: LeftInteractions -> Interactors -> Hand
• Confirm that there's HandPokeInteractor as a child element of Hand

• Click PalmMenu
• Drag and drop HandPokeInteractor to Element 0 directly under LeftHandedGameObjects in Dominant Hand Game Object Filter

[Register right hand fingertip object]

• Open OVRInteractionComprehensive child element of OVRCameraRig object
• Further open child elements in this order: RightInteractions -> Interactors -> Hand
• Confirm that there's HandPokeInteractor as a child element of Hand
• Click PalmMenu
• Drag and drop HandPokeInteractor to Element 0 directly under RightHandedGameObjects in Dominant Hand Game Object Filter

[Left Hand Settings]

In Left Hand of Dominant Hand Game Object Filter, you need to explicitly specify the Hand Ref associated with the left hand.

• Drag and drop PalmMenu to the area next to Left Hand
• When a dialog appears, select HandRef associated with LeftInteractions

4. Display Object Settings

When URP is selected, sample objects prepared may not display correctly due to material errors. Finally, adjust the sample objects.

• Create Material anywhere in the Assets folder
  This article sets the name as *CtrlObjMaterial
• Click ControlledObject child element of PalmMenu
• Drag and drop CtrlObjMaterial to the area marked Default-Material next to Element0 in MeshRenderer

5. Testing Operation

With the settings up to this point, you can operate the menu displayed on the left palm with fingers of the other hand. Also, menu display/hide can be toggled with tapping using the left thumb and index finger.

The PalmMenu prefab has pre-implemented object manipulation, but you can implement your own features by arranging this as a base. For details, refer to the Menu child element of PalmMenu.
For example, Unity's ScrollView is used for UI behavior, so you need to learn how to use this. Also, the PalmMenuExample script attached to the Menu object is helpful for more detailed behavior descriptions.

6. Bonus: Display Toggle by Palm Orientation

This time we toggled menu display with left thumb and index finger tapping, but you can also display the menu only when the palm faces toward you. I'll introduce this as an arrangement example of this sample.

Palm Orientation Detection Settings

• Pay attention to Menu child element of PalmMenu
• Disable OVR Button Active State
• Click Add Component at the bottom of Menu's Inspector
• Search for Transform Recognizer Active State and add
• Drag and drop LeftInteractions child element of OVRInteractionComprehensive to Hand in Transform Recognizer Active State
• Drag and drop Features child element of LeftInteractions to Transform Feature State Provider
• Change Transform Feature Configs to Palm Towards Face
• Open TransformConfig and set DefaultTransformFeatureStateThresholds.asset to Feature Thresholds *Search for DefaultTransformFeatureStateThresholds.asset in Project tab. (Select Search: All as search option)

Display/Hide Toggle Settings

• Find ActiveState Selector in Menu object's Inspector
• Drag and drop Menu object to Active State
• If the following dialog appears, select TransformRecognizerActiveState

• Find SelectorUnityEventWrapper in Menu object's Inspector
• Delete events set in When Unselected with - button
• Click + button for both When Selected and When Unselected
• Drag and drop Visuals child element of Menu object to None area in each event
• Click No Function in When Selected
• Click GameObject -> SetActive
• Check On
• Click No Function in When Unselected
• Click GameObject -> SetActive
• Check Off

Now the menu will only be displayed when the palm faces toward you.

7. Complete List of Meta XR SDK Articles

Meta XR SDK Series Index