Skip to main content
Conversational Avatar Communication

Tutorials

This page sequences a first, end-to-end run of the avatar call demo: two participants exchanging an MPEG ARF avatar over a call, built from the components and repositories described on the Implementation status page and Repositories on the project index.

ARF (ISO/IEC 23090-39) is not yet published as an International Standard (MPEG's report of its 155th meeting records it at Final Draft International Standard) and the reference tooling is still maturing (see Implementation status), so treat this as a sequencing guide rather than a finished, copy-paste recipe. It tells you which repository covers which part of the pipeline and in what order to bring them up. Exact build commands, dependencies, and configuration flags are not documented on this project's Implementation status page or repository list, so for those, follow the README in each repository (linked below and on Repositories).

What you'll end up with​

A base avatar loaded and bound in the Unity presentation, animated either from the sample recorded animation or from a live tracking feed, with the animation exchanged over the call path built by the Avatar Call repository.

Before you start​

  • Read Implementation status for the ARF split into the Base Avatar Format (the static asset: skeletons, meshes, blendshapes, skins, landmarks and nodes) and the Animation Stream Format (the Avatar Animation Units that drive it), and for how the tooling is organised into a tracking/animation producer, a transport/packaging path, sample content, and a Unity-based presentation.
  • Check Repositories for the current repository list and status.

1. Start from the sample content​

Repository: Avatar Content (rt-avatar-content)

This repository "holds reference MPEG ARF content for real time avatar communication": pre-built base avatars and recorded animation streams. Start here rather than with a live capture rig: it lets you validate the presentation and call path against known-good data before adding live tracking. This project's own docs do not document the exact file layout or how to load a sample asset, so check the repository's own README for that.

2. Bring up the call / transport path​

Repository: Avatar Call (rt-avatar-call)

This repository "supports the development of a reference framework for avatar communication", with the stated goal of demonstrating "WebRTC communication using the MPEG ARF and related animation compression". This is the transport/packaging component from the Implementation status page: it exchanges Animation Stream Format data between the two ends of a call. As with the sample content, this project's own docs do not spell out the exact build steps, dependencies, or how to run it (standalone service, Docker setup, or a library integrated into the Unity presentation): confirm all of that from the repository's README before proceeding.

3. Load and render the base avatar​

Per the Implementation status page, the Unity-based presentation loads a base avatar from its container, places it in a scene using the MPEG-I Scene Description avatar node (the MPEG_node_avatar glTF extension), and applies the incoming animation stream to render it. Neither the Implementation status page nor the repository list identifies a repository dedicated solely to this presentation role: check whether it ships as part of one of the repositories below (most likely Avatar Unity Tracker, since it is the only Unity-based repository listed) or is not yet published separately.

4. Add live tracking​

Repository: Avatar Unity Tracker (rt-avatar-unity-tracker)

This is a "Sample OpenXR tracker for ARF avatar animation in Unity". This is the tracking/animation producer component from the Implementation status page: it takes camera and tracking input, maps it onto the loaded avatar's landmarks and joints, and emits Animation Stream Format data. Once steps 1-3 work end to end with the sample content, swap this component in to drive the call from a live feed instead of a recording.

Where rt-3gpp-swap fits​

The 3GPP SWAP (rt-3gpp-swap) repository is also listed in the repository set for this project, but its own description only says it "is related to Conversational Avatar Real-Time Communications", without stating which of the four components above it covers. We could not confirm its role from this project's own docs, so it is not placed in the sequence above: check the repository directly if you need it.

Putting it together​

In order: get the sample content running through the Unity presentation and the call path first (steps 1-3), confirm the animation renders correctly end to end, then replace the recorded animation with the live tracker (step 4). Because ARF is not yet a final standard, pin every repository to a known, matching specification draft, as noted on Implementation status.

What isn't documented here yet​

This project's Implementation status page does not currently give exact CLI commands, Docker or build instructions, network ports, or configuration files for any of the repositories above. For those, use the README in each repository linked above, or the YouTube channel, which hosts practical videos from developers on the use of the Reference Tools generally.

Watch

Developer Exchange​

Video

Introducing the Avatar Real-Time Communication Project - Developer Exchange

Nils Duval (Motion Spell)

Browse all 5G-MAG videos →