XR/3D Scenes with MPEG-I Scene Description

Building and using XR Player on Meta Quest 3

This guide covers compiling the XR player sample Unity project for Meta Quest 3 and configuring it with specific glTF content.

What you will build: a native Meta Quest 3 application that loads glTF scenes with the MPEG glTF extensions, installed and running on the headset.

Prerequisites

• adb installed on the machine, and a Meta Quest 3 with developer options and USB debugging enabled and connected.
• Unity 3D 2022.3.34f1 with both the Android and iOS support modules installed (Android API 28 and NDK 27.2.12479018 are used).
• git LFS (required to clone the rt-xr-content test content).
• The native build tools used to compile the MAF library and media pipelines: Meson (the build system) and SWIG (used to generate the C# bindings). See the build steps below for how they are invoked.

This guide assumes a Windows environment with a git-bash terminal (eg. to run shell scripts); the same instructions apply to other platforms.

The tutorial covers the following steps:

1. Clone the XR Player unity project
2. Compile & install media pipeline library and plugins for Android
3. Configure the glTF asset to be loaded by the XR Unity player
4. Build and run the Unity project
5. Launch the player

Step 1: Clone the XR Unity Player project

rt-xr-unity-playerThe Unity project for the XR Player application.

git clone --recursive https://github.com/5G-MAG/rt-xr-unity-player.git

Note: --recursive is required to get all submodules checked out.

Step 2: Build and install media pipelines

Clone and install the source code

rt-xr-maf-nativeC++ Media Access Function library (libmaf) and the avpipeline media pipeline plugin.

git clone https://github.com/5G-MAG/rt-xr-maf-native.git
cd rt-xr-maf-native

Compile the MAF library and media pipeline plugins

This step builds libmaf (the C++ Media Access Function library used by the player), the avpipeline plugin (a media pipeline built on FFmpeg that decodes audio and video), and the C# bindings. The build uses Meson, and SWIG generates the C# bindings.

Compile and install dependencies

rt-common-sharedProvides the avcodec-build scripts used to compile the avpipeline (FFmpeg) dependencies.

Clone rt-common-shared as a sibling directory to rt-xr-maf-native, then follow these instructions to compile the avpipeline dependencies. The avcodec-build produces the FFmpeg libraries that the copy commands below move into the project; if it has not been run, those cp commands will fail because the source files do not exist.

Copy the generated libraries into the avpipeline subproject:

mkdir -p ./rt-xr-maf-native/subprojects/avpipeline/external/avcodec/android/arm64-v8a/lib
cp ./rt-common-shared/avcodec-build/build/ffmpeg/aarch64/lib/*.so ./rt-xr-maf-native/subprojects/avpipeline/external/avcodec/android/arm64-v8a/lib

cp -r ./rt-common-shared/avcodec-build/build/ffmpeg/aarch64/include ./rt-xr-maf-native/subprojects/avpipeline/external/avcodec/android/arm64-v8a/include/

cp ./rt-common-shared/avcodec-build/build/ffmpeg/aarch64/LICENSE ./rt-xr-maf-native/subprojects/avpipeline/external/avcodec/android/arm64-v8a/LICENSE

Configure cross compilation configuration

• Download and install the Android NDK. The next steps assume a Windows x86_64 environment with the Android NDK 27.2.12479018 installed in C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk.
• Locate the ./rt-xr-maf-native/crossfile/android-arm64-v8a and modify it to point to your local NDK installation, for instance:

[binaries]
ar = ['C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk\27.2.12479018\toolchains\llvm\prebuilt\windows-x86_64\bin\llvm-ar']
c = ['C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk\27.2.12479018\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android28-clang.cmd']
cpp = ['C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk\27.2.12479018\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android28-clang++.cmd']
c_ld = ['C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk\27.2.12479018\toolchains\llvm\prebuilt\windows-x86_64\bin\ld.lld']
cpp_ld = ['C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk\27.2.12479018\toolchains\llvm\prebuilt\windows-x86_64\bin\ld.lld']
strip = ['C:\Users\<your_user_name>\AppData\Local\Android\Sdk\ndk\27.2.12479018\toolchains\llvm\prebuilt\windows-x86_64\bin\llvm-strip']

Configure and compile the MAF library and media pipeline plugins

• libmaf and C# bindings are enabled by default.
• the avpipeline plugin needs explicit configuration, the avcodec_dir must point to a subdirectory of the project where platform specific dependencies have been copied:

cd rt-xr-maf-native
meson setup --wipe -Davpipeline=true -Davpipeline:avcodec_dir=external/avcodec/android/arm64-v8a build/android/arm64-v8a --cross-file crossfile/android-arm64-v8a
meson compile -C build/android/arm64-v8a

Install the media pipeline factory and plugins into the Unity project

Assuming rt-xr-unity-player repository has been cloned in a sibling directory ../rt-xr-unity-player, run the following commands with the correct path that applies to your installation:

export ANDROID_NDK_HOME='/c/Users/<your_user_name>/AppData/Local/Android/Sdk/ndk/27.2.12479018'
cd rt-xr-maf-native
scripts/install_android.sh ../rt-xr-unity-player/Packages/rt.xr.maf

The script copies the following:

• all compiled libraries from rt-xr-maf-native/build/android/arm64-v8a/ to rt-xr-unity-player/Packages/rt.xr.maf/bin/android/arm64/
• C# bindings source code from rt-xr-maf-native/subprojects/maf_csharp/swig/ to rt-xr-unity-player/Packages/rt.xr.maf/maf/swig/
• avpipeline dependencies from rt-xr-maf-native/subprojects/avpipeline/external/avcodec/android/arm64-v8a/lib/ to rt-xr-unity-player/Packages/rt.xr.maf/dependencies/ffmpeg/7.1/android/arm64 along with the related LICENSE
• libc++ from $ANDROID_NDK_HOME/toolchains/llvm/prebuilt/$ANDROID_NDK_HOSTNAME/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so to rt-xr-unity-player/Packages/rt.xr.maf/dependencies/libc++/android/arm64

Make sure all the *.so libraries are configured properly in the Unity Editor.

For each library, in the inspector panel:

• Android platform must be checked
• arm64 must be selected

The configuration is stored in Unity's *.meta sidecar files and are tracked in the Unity repository. Unity removes the *.meta files if the resource they reference is not found when opening a project.

Step 3: Configure the glTF asset to be loaded by the XR Unity player

rt-xr-contentTest glTF content (for example the Studio Apartment scene) used with the player.

Clone the rt-xr-content repository. This requires git LFS to be installed on your system.

git clone https://github.com/5G-MAG/rt-xr-content.git

Push glTF content to the Meta Quest 3, for instance the "Studio Apartment":

cd rt-xr-content
adb push ./studio_apartment /storage/emulated/0/Android/data/com.fivegmag.rtxrplayer/files/studio_apartment

Note: the target path is the application's private storage (Android/data/com.fivegmag.rtxrplayer/...), which is created when the app is first installed. If the push fails because the directory does not exist, install the application first (Step 4) and then push the content.

Open the rt-xr-unity-player directory as an existing project from Unity Hub.

Now configure the glTF asset in the Unity project by adding the path, e.g. /storage/emulated/0/Android/data/com.fivegmag.rtxrplayer/files/studio_apartment/studio_apartment.gltf

Step 4: Build and run the Unity project

In the Unity Editor:

1. Locate the File > Build Settings menu
2. Make sure that Android is the selected platform, Switch Platform if needed
3. Ensure that MetaQuestARF is the default scene.
4. Select the device on which the application will be installed.
5. Hit Build and Run to compile the project and install it on the mobile device

Step 5: Launch the player

Locate and launch the player in the Meta Quest 3.

Expected result: the player starts on the headset and renders the configured glTF scene (for example the Studio Apartment). If no scene appears, check that the content was pushed to the app-private path (Step 3) after the application was installed, and see Troubleshooting below.

You now have the XR Player running on Meta Quest 3. The app is installed on the headset and rendering the glTF scene you configured in Step 3.

Troubleshooting

If the player launches but XR input or rendering does not work, check the OpenXR configuration.

Symptom: the scene does not render in XR, or the headset is not detected. Ensure the project is configured for OpenXR with Meta Quest support:

Symptom: controllers or hand input do not respond. Ensure the Meta Quest interaction profiles are configured:

Next steps

• Try authoring your own content: Creation of MPEG-I Scene Description Test Assets.
• Explore the sharing use cases: Immersive and 3D Media message.
• Return to the Tutorials index.