Real-time conversational avatars, animated from audio and video, for immersive calls over 5G.
Avatar Communications covers real-time conversational avatar systems in 5G contexts, targeting the MPEG Avatar Representation Format (ARF) and its use for avatar-based communications. Avatars are synthesised digital representations of participants, animated in real time from audio and video input, enabling immersive video calls and virtual presence experiences. 5G-MAG's work looks at the tools needed to encode, render, and stream avatars over 5G networks: the avatar representation itself, its animation, and the media transport that carries it. For acronyms used here, see the Glossary.
The avatar representation reuses the MPEG-I Scene Description model: the MPEG_avatar glTF extension, described on the MPEG-I Scene Description page, is how a 3D avatar is placed and animated within a scene.
Key specifications: ISO/IEC 23090-39 (MPEG Avatar Representation Format, ARF), 3GPP TR 26.813 (Study of Avatars in Real-Time Communication Services, the FS_AVATAR study item, completed in Release 19).
Reference tools: The 5G-MAG software implementation is on the Developer Portal under Avatar Communications.
Go deeper
Technology & Analysis
The ARF data model, end-to-end procedure, and how avatars are placed in a scene via MPEG-I Scene Description.
Standards Tracking
The normative specifications (ARF, IVAS) and 5G-MAG's contributions on avatar communications standardisation.
Software Tools
The reference implementation for real-time conversational avatars over 5G.
The slide deck below introduces the Avatar Communications reference tools and their role in 5G real-time communication.
Slide deck: Avatar Communications reference tools.
Technical Documentation
General information about Avatar Communications
An avatar communication system combines three pieces: a representation of the avatar (its 3D model and how it is described), an animation stream that drives it in real time from a participant's audio and video, and a media transport that carries these over the network so remote participants see and hear the avatar. 5G-MAG's work looks at how these pieces map onto 5G media delivery. The introductory material below gives a first orientation.
The ARF data model
The Avatar Representation Format (ISO/IEC 23090-39) is built from two complementary specifications:
- A Base Avatar Format that stores the avatar asset. Its data model is composed of skeletons, meshes, blendshapes, skins, landmarks, and nodes (joints). The skeleton and joints give the avatar its articulation; meshes and skins give it surface geometry and how that geometry deforms with the skeleton; blendshapes provide expression and fine facial deformation; landmarks give named reference points used to map tracking data onto the model.
- An Animation Stream Format that carries the time-varying data used to animate a base avatar. The stream is a sequence of Avatar Animation Units (AAUs). Each AAU is a self-contained packetisation unit, similar in role to a NAL unit in video coding, and consists of an AAU header followed by zero or more AAU packets. Each AAU references an Avatar ID so that, in a multi-party session, animation data can be routed to the correct avatar.
The base avatar can be stored and exchanged in two container types: an ISOBMFF container (ISO/IEC 14496-12) or a Zip-based container (ISO/IEC 21320-1). Because the base avatar and the animation are separate, a client can fetch and load the model once (or reuse one it already holds) and then receive only the compact animation stream during the call.
Scene placement via MPEG-I Scene Description
ARF does not, by itself, position an avatar in a room; that is the job of MPEG-I Scene Description (ISO/IEC 23090-14), a set of extensions to Khronos glTF 2.0. Avatar support was added to Scene Description through an amendment (ISO/IEC 23090-14:2023 Amendment 2), which defines a glTF node extension that marks a particular node as the root/skeleton node of a humanoid avatar. In the 5G-MAG documentation this is referred to as the MPEG_avatar glTF extension, described on the MPEG-I Scene Description page. The Presentation Engine reads the scene, resolves the avatar node, binds the ARF base avatar to it, and then applies the incoming animation stream frame by frame. This is the same Presentation Engine / Media Access Function decoupling used across the other MPEG-I Scene Description tooling.
End-to-end procedure
At a high level, a conversational avatar session proceeds as follows:
- Provisioning. Both endpoints obtain the base avatar (either a stored ARF asset or one generated for the user). Because the avatar model is shared, only animation and audio need to flow during the call.
- Session setup. The endpoints negotiate a real-time session. In a 5G context this uses the IMS-based conversational path (MTSI, TS 26.114) or the RTC framework (TS 26.506); the immersive audio codec (IVAS) and any avatar animation stream are negotiated through SDP.
- Capture and tracking. The sender tracks the participant from camera and microphone input and derives animation parameters (skeletal pose, facial blendshape weights, and similar) mapped onto the ARF landmarks and joints.
- Encode and packetise. Animation parameters are formatted as ARF AAUs; audio is encoded with IVAS.
- Transport. AAUs are carried over RTP. The IETF is defining an RTP payload format for ARF animations (draft-ietf-avtcore-rtp-avatar) with single-unit, fragmentation, and aggregation packet modes; IVAS audio uses its own RTP payload format. The IMS data channel is one candidate carrier for the animation data.
- Render. The receiver's Presentation Engine applies the animation stream to the bound base avatar within the scene, and IVAS renders the spatial audio so the avatar is heard from its scene position.
Interfaces and interoperability
An implementer has to respect four interoperability boundaries: the ARF base-avatar asset (so any conforming client can load the same avatar), the ARF animation stream and AAU structure (so animation produced by one endpoint can drive an avatar at another), the scene-description document that anchors the avatar, and the session-level negotiation (SDP media descriptions for IVAS audio and for the avatar animation payload). Keeping animation production separate from rendering means the tracking/animation component and the Unity-based presentation can evolve independently, provided both sides agree on the ARF stream format.
ARF (ISO/IEC 23090-39) was still at the Draft International Standard stage as of 2025, so the base-avatar and animation-stream details above can change before final publication. Confirm the current stage on the ISO catalogue page and treat reference tooling as tracking a moving target.
The following points on this page were not confirmed against a primary source and should be checked before publication:
- TS 26.114 (MTSI) and TS 26.506 (RTC over 5G) as the delivery path for avatar animation, and the exact title of TS 26.506; confirm placement and titles against the 3GPP SA4 work plan.
- ISO/IEC 21320-1 as the Zip-based container profile and the specific ISO/IEC 14496-12 edition referenced by ARF.
- The claim that the avatar animation stream is negotiated at session level via SDP alongside IVAS, and that the IMS data channel carries the animation. These describe candidate mechanisms and should be checked against the current ARF, IETF, and 3GPP drafts.
The avatar study number (TR 26.813, "Study of Avatars in Real-Time Communication Services") was confirmed via independent web search cross-checking two sources. Established citations (ISO/IEC 23090-39, ISO/IEC 23090-14 and Amendment 2, glTF 2.0, and the IETF ARF RTP payload draft) are not part of this caution.
Related
- XR and MPEG-I Scene Description: the wider XR area, including the scene-description model that positions and animates avatars.
- MPEG-I Scene Description: the
MPEG_avatarglTF extension used for 3D avatar representation. - Standards: Avatar Communications: the standards-tracking view of this topic.
Refer to the Tech repository to contribute to this documentation.