Skip to main content
Testbeds

VOPS Conformance Validator

At a glance
  • What it is: the conformance validator of 3GPP TS 26.265 (Video Capabilities and Operation Points, V19.2.0, Release 19), annex B.2.3. It checks whether an AVC or HEVC bitstream conforms to a video operation point.
  • What it covers: the seven operation points of TS 26.265, from AVC HD to MV-HEVC stereo.
  • What it is not: an encoder or decoder. It tests bitstreams that encoders produce.
  • Status: under development. Every operation point is implemented; each still needs testing with its conformance bitstreams.

What it does​

TS 26.265 defines service-independent video operation points and capabilities, which 3GPP service specifications and third-party services may reference. An operation point combines a video coding format with a video representation format, and a receiver conforming to it can decode the bitstream and render its video signal.

A video signal enters the Video Encoder, configured through VID-ENC-API, which produces coded metadata and a video bitstream. In the receiver conforming to the operation point, the Video Decoder, configured through VID-DEC-API, passes the video signal and rendering metadata to Rendering. An Application spans both.
Reference architecture for video Operation points and capabilities. Redrawn from TS 26.265 V19.2.0, figure 4.2-1.

Where the validator fits​

TS 26.265 annex B provides a set of conformance functions: annotated video test signals, reference encoders and configurations, a conformance validator, conforming decoders, and conformance bitstreams. The validator tests whether a bitstream claiming to conform to an operation point does, and if not, which test assertions fail. This testbed is that validator.

The Encoder Configuration for Operation Point and the Video Signal Test Sequences with annotation feed the Reference Software Encoder. Its bitstream goes to the Conformance Validator, which returns a report and a description; coded metadata goes to the Conforming Decoder, which returns a report and an MD5. All of these, with the encoding configuration and the video signal metadata, form the Conformance Bitstream with annotation, which provides a bitstream to a decoder under test and conformance information to the decoder conformance test.
Overview of conformance functionalities provided for this specification. Redrawn from TS 26.265 V19.2.0, figure B.1-1.

Operation points​

The operation points of TS 26.265 (table 6.1-1), and how the validator names them.

Operation pointBitstreamVideo formatIn the validator
3GPP-AVC-HDAVC High Progressive Profile, Level 4.03GPP-HD3GPP-AVC-HD
3GPP-HEVC-HDHEVC Main 10 Profile, Main Tier, Level 4.13GPP-HD3GPP-HEVC-HD
3GPP-HEVC-HD-HDRHEVC Main 10 Profile, Main Tier, Level 4.13GPP-HDR3GPP-HEVC-HDR
3GPP-HEVC-UHD-HDRHEVC Main 10 Profile, Main Tier, Level 5.13GPP-HDR3GPP-HEVC-UHD, 3GPP-HEVC-UHD-HDR
3GPP-HEVC-StereoHEVC Main 10 Profile, Main Tier, Level 5.2, frame packed3GPP-Stereo3GPP-HEVC-Stereo
3GPP-MV-HEVC-Main-StereoMV-HEVC, two layers3GPP-Stereo3GPP-MV-HEVC-Main-Stereo
3GPP-MV-HEVC-Ext-StereoMV-HEVC, two layers3GPP-Stereo3GPP-MV-HEVC-Ext-Stereo

TS 26.265 itself also uses "3GPP-HEVC-HDR" and "3GPP-HEVC-UHD" in its clause text for the HDR and UHD operation points.

How it works​

The validator parses the bitstream into an XML description of its NAL units and parameters, then checks that description against XSD 1.1 schemas, one set per operation point, and reports every passing and failing assertion. A "shall" requirement that fails is reported as an error, a "should" as a warning.

The bitstream (.264 or .hevc) goes to the AVC or HEVC parser, which generates an XML document. The validator (XSD 1.1) checks it against the XSD schemas of the operation point and produces the validation report, with every assertion passed or failed.
How the validator works, from its README (validation pipeline, steps 1 to 4).

TS 26.265 leaves the semantics and format of the validator report for further study, so the JSON report is the tool's own.

Run it
cd ~
git clone https://github.com/5G-MAG/vops-conformance-validator.git
cd vops-conformance-validator
python -m pip install -r requirements.txt

Validate a bitstream against an operation point, with a JSON report:

# Validate bitstream against predefined operation point with JSON report
$ python -m sa4_bitstream_validator validate bitstream_path 3GPP-MV-HEVC-Main-Stereo --report report.json

dump writes the XML description of a bitstream, and check tests a description against one or more schemas. config.yaml maps each operation point to its schemas in bitstream_rules/. The repository README lists every option.

Repositories​

Detailed repository metadata is not curated for this project yet, see the repository list above.

AreaDetails
ScopeWhat the validator covers and its operation points: Scope
TutorialsInstall and run it: Tutorials
ResourcesRepository and releases: Resources
Technical documentationVideo Operation Points on the Tech portal
StandardsVideo Operation Points on the Standards portal

See Standards: VOPS Conformance Validator for the specification.

Sources for this page
  • At a glance and What it does: TS 26.265 V19.2.0, Introduction, clause 4.2 (Operation Point, Conforming Receiver) and annex B.2.3.
  • Where the validator fits: TS 26.265 V19.2.0, annex B.1 and B.3.
  • Operation points: TS 26.265 V19.2.0, table 6.1-1, clauses 6.2.2.2, 6.3.2.2 to 6.3.5.2, 6.3.6 and 6.3.4.3; validator names from the repository README.
  • How it works: the repository README (including the pipeline figure); TS 26.265 V19.2.0, annex B.2.3.
  • Figures: redrawn from TS 26.265 V19.2.0, figures 4.2-1 and B.1-1.
  • Status: the repository README (status badge and operation point table).