Introduction
This tutorial provides the instructions to set up the MPEG V3C Immersive Platform for an Android device using content streamed from a DASH server.
What you will build: a Unity-based V3C player running on an Android phone that decodes volumetric content and plays it back from a Dynamic Adaptive Streaming over HTTP (DASH) server you run on your own machine.
This tutorial pulls together four repositories:
Prerequisites
This walkthrough was done on Windows, building an APK for an Android phone (a Samsung S21 in this example). For the supported and tested Android devices, see the Requirements page. Before you start, install the tools referenced in the steps below:
Unity Hub is the easiest way to install the pinned Unity version. The exact download links and versions for each tool are given inline in Step 0 below.
Step 0: Setting up the environment
This walkthrough installs and builds the platform on Windows. The target device to install the apk with the Unity Player will be an Android phone (in this example a Samsung S21).
As indicated in the instructions in the rt-v3c-unity-player, this tutorial uses Unity 6000.0.25f1. Download Unity Hub and select the version of Unity to install, which can also be downloaded from here: Unity 6000.0.25 release notes
As indicated in the instructions in the rt-v3c-decoder-plugin, the following are also needed:
- CMake (for example 3.30.4) - Download from Cmake
- Android Studio - Download from Android - Once downloaded make sure you install NDK r27c (27.2.12479018), and API 35.
- Docker Desktop - Download from Docker - this is needed to execute Docker in Windows.
Step 1: Clone the Unity Player Repository
rt-v3c-unity-player Unity project for the Android V3C player.Clone the Unity player repository into your home directory:
cd ~
git clone https://github.com/5G-MAG/rt-v3c-unity-player
Although not immediately used, this will create the directories where the decoder plugin files will be copied.
Step 2: Install the Decoder Plugin
rt-v3c-decoder-plugin Native V3C decoder plugin, built for Android in this step.Clone the repository:
cd ~
git clone --recurse-submodules https://github.com/5G-MAG/rt-v3c-decoder-plugin.git
Install the dependencies:
cd ~/rt-v3c-decoder-plugin
./Scripts/dl_deps.sh
Add the additional dependency regarding avcodec libraries. For this, instructions are provided in rt-common-shared. Use the Docker build instructions with Git Bash.
cd ~
git clone --recurse-submodules https://github.com/5G-MAG/rt-common-shared.git
cd ~/rt-common-shared/avcodec-build/
docker build -t ffmpeg-builder:27 --build-arg NDK_VERSION=27.2.12479018 .
docker run -v /$(PWD)/build/ffmpeg/aarch64:/usr/build/ffmpeg --env TARGET_ABI=aarch64 --env ANDROID_API_LEVEL=35 ffmpeg-builder:27
In Windows, the build artifacts can be found in your user folder rt-common-shared\avcodec-build\build\ffmpeg\aarch64.
Because this build targets an Android device, once compiled, manually copy the .so libraries from rt-common-shared\avcodec-build\build\ffmpeg\aarch64\lib into the rt-v3c-decoder-plugin\External\avcodec\7.1\Android\arm64-v8a\lib directory. Create the folders under the rt-v3c-decoder-plugin: External\avcodec\7.1\Android\arm64-v8a\lib.
Make sure the ndk is available in the Android directory.
At this point, back in the directory of the rt-v3c-decoder-plugin, compile for Android:
cd ~/rt-v3c-decoder-plugin
./Scripts/build_android.sh release all
Copy the plugins into the rt-v3c-unity-player directories:
cd ~/rt-v3c-unity-player
../rt-v3c-decoder-plugin/Scripts/copy_libs.sh ./Packages/V3CDecoder/Runtime/Plugins release Android
Step 3: Building the Unity Player for Android
In Unity Hub, import the "rt-v3c-unity-player/V3CImmersiveTest" project. Make sure that the scene V3C-Simple Player.unity is loaded.
Go to File->Build Settings (or Ctrl+Shift+B), select the Android target and select either Build or Build and Run.
Unity will generate a .apk file that you can copy and install on your phone. To use the Build&Run feature, you will need to activate the Debug through USB feature on your device (requires developer mode).
Note that you will need to manually install the content and/or configuration files on your device. Run the application once on your device to ensure the creation of the folder.
Step 4: Importing configuration files
rt-v3c-content Sample V3C test content for on-device playback and streaming.With the application already installed on the Android device and the directories automatically created, download the V3C Content for testing locally on the device and for streaming.
git clone --recurse-submodules https://github.com/5G-MAG/rt-v3c-content.git
The content of the folder "on-device-data" can be copied into the device (in this example: "\Internal storage\Android\data\com.InterDigital.V3CSimplePlayer\files").
Step 5: Setting up a DASH streaming server
rt-common-shared Includes the simple express server used here as the DASH streaming server.Before copying the content in the device, you can modify the config.json file to insert the IP address of the machine hosting the DASH server and the port.
Use the simple express server available in rt-common-shared.
Just install the server following the instructions, then copy the content of the "on-server-data" inside the public folder of the DASH server. Make sure to unzip the packages containing the segments and mpd for each test sequence.
Streaming can be started from the Unity player.
Troubleshooting
If you experience any errors during the Unity player build. Try selecting the V3CDecoder tab on the top menu and click on "Check plugins" and "Check settings".
Next steps
- Browse more walkthroughs and recorded talks in the Tutorials and Developer Exchange.
- See the source repositories behind each component on the Resources page.
- For background on the specifications and pipeline, read the Scope page.