Skip to main content
Tech

XR and MPEG-I Scene Description

Interactive, spatially-aware 3D scenes and immersive media delivered over 5G Media Streaming.

Extended Reality (XR) and 3D scene delivery bring interactive, spatially-aware content to 5G devices, from augmented reality overlays to full virtual environments. 5G-MAG's work centres on MPEG-I Scene Description (ISO/IEC 23090-14), a standard format for describing dynamic 3D scenes composed of glTF (GL Transmission Format) 3D objects, audio, and haptic elements. Scene Description content can be transported using the 5G Media Streaming (5GMS) framework (3GPP TS 26.512), enabling adaptive and policy-driven delivery of immersive media. 5G-MAG also covers Avatar Communications, addressing real-time avatar representation and interaction in 5G contexts. For acronyms used here, see the Glossary.

Start here

New to this area? Read the Overview on MPEG-I Scene Description first for the scene-description model, then see Avatar Communications for real-time avatar work.

Key specifications: ISO/IEC 23090-14 (MPEG-I Scene Description), 3GPP TR 26.928 (Extended Reality (XR) in 5G), TS 26.512 (5GMS transport), ISO/IEC 23090-2 (OMAFv2, the second edition of the Omnidirectional Media Format for omnidirectional/360-degree media).

Reference tools: The 5G-MAG software implementation is on the Developer Portal under XR Media Integration in 5G.

Go deeper​

Technology & Analysis

The scene-description model, runtime, and device architecture. Closely related to Avatar and Volumetric Video.

Standards Tracking

The normative specifications and 5G-MAG's contributions.

Software Tools

The reference implementation for MPEG-I Scene Description delivery.

Execution Plan

The slide deck below introduces the XR Media reference tools and how MPEG-I Scene Description is used in 5G delivery.

Slide deck: XR Media and MPEG-I Scene Description reference tools.

Download the Slidedeck

Technical Documentation​

The following resources are available:

General information about XR and MPEG-I Scene Description​

Technical foundations​

XR delivery combines a scene format, a runtime that plays it, a delivery layer over 5G, and a device capability model. This section explains how they connect; the child page goes deeper on the scene format and its extensions.

The scene format and its runtime model​

MPEG-I Scene Description (ISO/IEC 23090-14) extends Khronos glTF 2.0 (also published as ISO/IEC 12113). glTF describes the static scene: node hierarchy, meshes, materials, textures, cameras and animations. The MPEG extensions add what glTF alone cannot express: references to external and timed media, circular buffers for streaming, spatial audio, real-world anchoring, interactivity, avatars, lighting and haptics.

At runtime, ISO/IEC 23090-14 defines a processing model with two roles:

  • The Presentation Engine parses the scene, renders each frame, and drives playback. It knows the viewer's pose and object poses and can pass that information downstream so delivery can be optimised (for example, fetching only the parts of a scene that are visible).
  • The Media Access Function (MAF) is asked, through the MAF API, to make a media object available in a given format. It builds a pipeline (access, demux, decode, format conversion) appropriate to the object's MIME type and codec parameters, and writes the result into buffers that the Presentation Engine reads. The MAF is deliberately transport- and codec-agnostic: the same scene can be served over MPEG-DASH, RTP/SRTP or local files without changing the scene document.

This decoupling is the core interoperability contract of the standard. The MPEG-I Scene Description and glTF 2.0 extensions page details the extensions and the buffer/MAF interfaces.

Delivery over 5G​

3GPP profiles these formats for delivery. For streamed and on-demand immersive media, the 5G Media Streaming (5GMS) framework applies: TS 26.501 defines the architecture (Media Session Handler, Media AF/AS, provisioning and reporting) and TS 26.512 defines the protocols and APIs. Scene-description content and its referenced media objects are treated as media that 5GMS can provision, deliver adaptively and report on. For conversational and low-latency AR (for example AR calls), the Real-Time Communication work (the TS 26.506/RTC family) applies, and split rendering (below) uses WebRTC transport.

Device architecture and capabilities​

TR 26.928 (Rel-16) and TR 26.998 (Rel-17) established the reference architectures for XR and for glass-type AR/MR devices, and the STAR (stand-alone) and EDGAR (edge-dependent) device classes. TS 26.119 (MeCAR) turns these into a concrete capability model: device categories (thin AR glasses, AR glasses, XR phone, XR HMD) and the audio, video, scene and XR-system capabilities each supports. MeCAR aligns the on-device XR client with the Khronos OpenXR runtime API, so an AR application can query poses, spaces and inputs in a portable way. It also references MPEG-I Scene Description (and optionally V3C, ISO/IEC 23090-5) as the media it must be able to consume.

Split rendering​

A STAR device can render a complex scene locally. An EDGAR device cannot, so rendering is split: the device sends its pose (and inputs) to a network renderer (edge or cloud), which produces the matching view and streams it back with low latency. Because a network round trip is visible as latency, the device applies pose-correction (for example asynchronous time-warp) to the received frame using the latest local pose.

The Split Rendering Media Service Enabler (TS 26.565, Rel-18) specifies this for non-IMS services. A Split Rendering Client (SRC) and a Split Rendering Server (SRS) establish a session using the SWAP control protocol: a configuration message carries the client's split-rendering configuration to the server, and a rendering-description message carries the description of the split-rendered media back to the client. The rendered media itself is carried over WebRTC. Immersive audio for split-rendering scenarios is handled by a separate set of SA4 deliverables (see the Standards view).

References to verify

These identifiers on this page were not confirmed against a primary source (the 3GPP/ETSI portals block automated access): TS 26.506 as the RTC general specification; and the current release placement and status of TS 26.119 (MeCAR) and TS 26.565 (Split Rendering Media Service Enabler). Verify against the 3GPP/ETSI work plan before publication.

note

Refer to the Tech repository to contribute to this documentation.