Skip to main content
Technical Analysis

MPEG-I Scene Description

At a glance
  • What it is: the MPEG standard that defines the structure and composition of a 3D scene and its assets, built on glTF 2.0.
  • Why it matters: it adds timed and immersive media to glTF, so a Presentation Engine can play scenes that a traditional 2D player cannot.
  • In this section: the reference architecture, glTF extensions, timed media, the MAF API and buffers, editions, and 3GPP delivery.
warning

This documentation is currently under development and subject to change. It reflects outcomes elaborated by 5G-MAG members. If you are interested in becoming a member of the 5G-MAG and actively participating in shaping this work, please contact the Project Office

Overview - MPEG-I Scene Description and glTF 2.0 Extensions​

This page explains the MPEG-I Scene Description model that underpins 5G-MAG's XR work: what it is, the reference architecture that plays back a scene, and the glTF 2.0 extensions that add timed and immersive media. For the wider XR context and reference tools, see the parent XR/3D Scenes with MPEG-I Scene Description page.

MPEG-I Scene Description (MPEG-I SD, ISO/IEC 23090-14) is a standard developed by MPEG (the Moving Picture Experts Group) for immersive media. It defines the structure and composition of a 3D scene, referencing and positioning 2D and 3D assets so the scene can be rendered properly, with support for real-time media delivery and interactivity.

MPEG-I SD builds on glTF (GL Transmission Format), which was chosen as the base scene-description technology. Developed by the Khronos Group, glTF is an open interoperable 3D asset format with a JSON scene description covering node hierarchy, meshes, materials, cameras and animations, and is designed to be usable regardless of the underlying 3D graphics API. The MPEG-I Scene Description extensions are listed as vendor extensions (MPEG_media, MPEG_scene_dynamic and others) in the Khronos glTF extensions registry, as read on 18 October 2026.

Reference architecture​

MPEG-I SD defines the following reference architecture. It shows how a scene flows from the media source, through the Media Access Function (MAF) and Buffers, to the Presentation Engine that renders it. The figure below sets out these components and their connections.

MPEG-I Scene Description reference architecture: the Presentation Engine requests media through the MAF API from the Media Access Function, which fills the Buffers that feed rendering.
MPEG-I Scene Description reference architecture: Presentation Engine, Media Access Function (MAF) and Buffers.
  • The Presentation Engine replaces a traditional 2D media player. It is responsible for multi-modal rendering of a scene composed of audiovisuals and haptics media. It also provides information about the viewer's and object pose to optimize delivery.
  • The Presentation Engine delegates media handling to the Media Access Function (MAF), which is responsible for media access and processing. The MAF constructs a suitable media pipeline to transform media from a delivery format into the formats used during rendering, using MIME type and codec parameters (the standard content-type and codec identifiers carried with each media object) to work out which formats it must support and assemble the correct pipeline.
  • The MAF API is used by the Presentation Engine to request immersive media in the scene.
  • The Buffers are fed by the processed media with a format defined through the scene description document in ISO/IEC 23090-14

Vendor extensions to Khronos glTF 2.0​

A glTF extension is a named addition to the base glTF format that lets vendors describe capabilities the core format does not cover; a renderer that does not understand an extension can ignore it unless the asset lists it in extensionsRequired (Khronos glTF 2.0 section 3.12; Khronos glTF 2.0 section 5.17.2). MPEG defines several such extensions, shown grouped by purpose in the figure below.

The figure groups the extensions by colour: the green group adds the timed framework, the blue group adds dynamic and temporal media, and the purple group adds real-time immersive and interactive media. The three groups are also listed in prose below, so the grouping does not depend on the colours.

MPEG vendor extensions to glTF 2.0, grouped by colour: green for the timed framework, blue for dynamic and temporal media, and purple for real-time immersive and interactive media.
MPEG vendor extensions to glTF 2.0, grouped by purpose (green: timed framework; blue: dynamic and temporal media; purple: real-time immersive and interactive media).

A first set of extensions (green in the figure) enable the timed framework including:

  • MPEG_media, which enables the referencing of external media streams that are delivered over protocols such as RTP/SRTP, MPEG-DASH, or others
  • MPEG_accessor_timed, used in a scene that contains timed media and/or metadata to describe access to the dynamically changing data
  • MPEG_buffer_circular, to extend the buffer into a circular buffer

A second group of extensions (blue in the figure) enables the inclusion of dynamic and temporal media including:

A third group of extensions (purple in the figure) enables the distribution of real-time immersive and interactive media content including:

  • Augmented Reality anchor (MPEG_anchor), to support AR experiences where virtual content is inserted into the user's real environment
  • Interactivity (MPEG_scene_interactivity, MPEG_node_interactivity), to describe interactivity at runtime with support for interactions between user and virtual objects and between virtual objects, with triggers based on proximity, visibility, collision or user input.
  • Avatar (MPEG_node_avatar), to support the representation of 3D avatars.
  • Lighting (MPEG_lights_texture_based, MPEG_light_punctual), to provide a realistic user experience including shadows and lighting.
  • Haptics (MPEG_haptic), to support haptics based on the MPEG standard for Coded representation of Haptics by attaching haptic information to a node or to a mesh.
  • MPEG-I media codecs (MPEG_primitive_V3C, MPEG_sampler_YCbCr), defined in annex G: the first provides the syntax of V3C objects (ISO/IEC 23090-5 or ISO/IEC 23090-12) as an extension to mesh.primitive, the second lets a video texture in YCbCr format be sampled natively on devices such as GPUs.

How timed media reaches the renderer​

The three extension groups above work together to bring streaming media into an otherwise static glTF scene. The mechanism is worth following end to end, because it is the part of ISO/IEC 23090-14 that differs most from plain glTF.

  • Declaring the source. MPEG_media declares one or more media items at the top level of the glTF document, each with one or more alternatives (for example the same content at different bitrates or in different codecs) and a URI. The URI can point to an MPEG-DASH MPD, an RTP source, a local file, or another supported source. This is a declaration, not a fetch; nothing is retrieved until the Presentation Engine asks for it.
  • Binding into the scene. An asset in the scene references that media indirectly. A video texture uses MPEG_texture_video to bind a glTF texture to a track of an MPEG_media item; a spatial audio source uses MPEG_audio_spatial; a dynamic geometry stream binds through the timed accessor. The binding always goes through a glTF accessor, so the renderer reads timed data using the same accessor concept it already uses for static vertex data.
  • Timed accessors and circular buffers. A normal glTF accessor points at a fixed region of a buffer. MPEG_accessor_timed marks an accessor whose contents change over time, and MPEG_buffer_circular turns the backing buffer into a ring buffer with multiple frames in flight. Each buffer frame carries a header (timestamp and length) so the Presentation Engine can pick the frame that matches the current presentation time and detect underflow or stale data. This is how the renderer stays synchronised with a stream it does not itself decode.

The MAF API and the buffer interface​

The split between the Presentation Engine and the Media Access Function is defined by two interfaces.

  • The MAF API is called by the Presentation Engine to initialise, start, update and stop a media pipeline for a referenced media object. On initialisation the Presentation Engine passes the media information from MPEG_media and the buffer configuration; it can also pass view information (viewer and object pose) so the MAF can optimise fetching, for example by requesting only visible tiles or a viewport-appropriate quality. The MAF selects one of the declared alternatives, builds the access/decode pipeline, and reports the format it will write.
  • The buffer API governs the buffers between the two roles. The MAF writes decoded, render-ready data into the buffers (respecting the format the scene requested); the Presentation Engine reads from them at render time. Circular buffers decouple the two rates, so jitter in access and decoding does not stall rendering as long as the buffer stays filled.

The result is that the Presentation Engine never needs to know the transport or codec of a media object. It knows only the scene, the accessors, and the buffer contract. Anything from a DASH-delivered video texture to an RTP-delivered point cloud looks the same from the renderer's side, which is what makes a single scene document portable across delivery methods and devices.

Editions and amendments​

This page follows ISO/IEC 23090-14:2025, second edition, published 2025-11. Its foreword states that it "cancels and replaces the first edition (ISO/IEC 23090-14:2023)" and "incorporates the Amendment ISO/IEC 23090-14/Amd 1:2023", and lists the main changes as "addition of the support of MPEG-I immersive media codecs, AR anchoring, interactivity, avatar, lighting and MPEG-I haptics". TS 26.143 V19.1.0 cites the AR, interactivity, avatar, lighting and haptics support as "ISO/IEC 23090-14 AMD 2".

Amendment 1 to the second edition, "Support of MPEG-I audio, scene understanding and other extensions", is at final draft stage (FDAM 1, dated 2026-05-01). Among its changes, it adds a recommendedSpatialComputingConfig object to the MPEG_anchor extension (clause 8.1).

A given implementation may support only a subset of the extensions. The developer-facing support matrix for the 5G-MAG reference tools is on the XR Implementation Detail page.

Relationship to 3GPP delivery​

MPEG defines the scene format; 3GPP defines how it is delivered over 5G and what a device must be able to play. Scene-description content and its referenced media can be carried using the 5G Media Streaming framework (3GPP TS 26.501 architecture, TS 26.512 protocols) for streamed cases, and over Real-Time Communication transports for conversational and split-rendering cases. On the device side, 3GPP TS 26.119 (MeCAR) references MPEG-I Scene Description as media that an AR device must be able to consume, and aligns the on-device client with the Khronos OpenXR runtime API (TS 26.119 V19.1.0 clause 9.2; TS 26.119 V19.1.0 Annex B.2). For that wider delivery and device context, see the XR/3D Scenes with MPEG-I Scene Description Tech page and the Standards view.

How it works​

XR delivery combines a scene format, a runtime that plays it, a delivery layer over 5G, and a device capability model (TS 26.119 V19.1.0 clause 5.2; TS 26.119 V19.1.0 clause 5.3).

Key specifications: ISO/IEC 23090-14 (MPEG-I Scene Description), 3GPP TR 26.928 (Extended Reality (XR) in 5G), TS 26.512 (5GMS transport), ISO/IEC 23090-2 (OMAFv2, the second edition of the Omnidirectional Media Format).

MPEG-I Scene Description (ISO/IEC 23090-14) extends Khronos glTF 2.0 (also published as ISO/IEC 12113). glTF describes the static scene: node hierarchy, meshes, materials, textures, cameras and animations (Khronos glTF 2.0 section 2.4; Khronos glTF 2.0 section 2.3; TS 26.119 V19.1.0 clause 2; TS 26.119 V19.1.0 clause 9.2).

For streamed and on-demand immersive media, the 5G Media Streaming (5GMS) framework applies: 3GPP TS 26.501 defines the architecture (Media Session Handler, Media AF/AS, provisioning and reporting) and TS 26.512 defines the protocols and APIs. For conversational and low-latency AR (for example AR calls), the Real-Time Communication work (the TS 26.506/RTC family) applies, and split rendering uses WebRTC transport.

On the device side, TR 26.928 (Rel-16) and TR 26.998 (Rel-17) established the reference architectures for XR and for glass-type AR/MR devices, and the STAR (stand-alone) and EDGAR (edge-dependent) device classes (TR 26.998 V19.0.0 clause 4.2.2; the Release 16 and 17 origins follow the change history of TR 26.928 V19.0.0 and TR 26.998 V19.0.0). TS 26.119 (MeCAR) turns these into a concrete capability model, device categories (thin AR glasses, AR glasses, XR phone, XR HMD; TS 26.119 V19.1.0 clause 10) and the audio, video, scene and XR-system capabilities each supports, and aligns the on-device XR client with the Khronos OpenXR runtime API, so an AR application can query poses, spaces and inputs in a portable way. A STAR device can render a complex scene locally; an EDGAR device cannot, so rendering is split between the device and a network renderer. The Split Rendering Media Service Enabler (TS 26.565, Rel-18) specifies this: a Split Rendering Client and Server establish a session over the SWAP control protocol (TS 26.565 V19.0.0 clause 8.2), and the rendered media itself is carried over WebRTC (TS 26.565 V19.0.0 clause 8.5).

Later releases: TS 26.119 and TS 26.565 were created in Release 18 (MeCAR and SR_MSE; TR 21.918 clauses 14.2 and 14.7). In Release 19, TS 26.119 V19.1.0 assigns IVAS levels to device types (CR 0003), TS 26.565 V19.0.0 carries the change request "Enhancements to SR_MSE" (CR 0003), and the work item SR_IMS adds TS 26.567, split rendering over IMS, with its own functional entities, metadata, metrics and session procedures (TR 21.919 clause 12.5). The change histories of TS 26.119 V19.1.0 and TS 26.565 V19.0.0 are in their annexes C and D.

5G-MAG's own reference tools overview slide deck introduces the XR Media reference tools and how MPEG-I Scene Description is used in 5G delivery.