Skip to main content
AI Traffic Characterization

Metrics

At a glance
  • Four categories (clause D.4.1, table D.4.1-1): A, traffic characteristics; B, response time and HTTP errors; C, service metrics for agents and computer use; D, task-related QoE.
  • Two vantage points: application-layer records logged per request, and packet captures parsed per packet and per flow.
  • What the study groups need: TR 26.870 clause 6.3.3.3 (table 6.3.3.3-1) maps each group of traffic characteristics it lists for other working groups (volume, temporal structure, latency, reliability, adaptivity) to the analysis in annex C and to these metrics.
  • Open point: an Editor's Note in annex B.1 says that whether and how TTFT, TTLT and tool-call latency can be used for AI traffic evaluation is for further study.

The categories​

Clause D.4 is a reference for the metrics produced by the testbed; the TR says its implementation details are given for completeness and to help understand the metrics reported. Clause D.4.2 shows the metrics collection pipeline (figure D.4.2-1), drawn on the overview page. The names in the tables below are the output keys of the testbed.

CategoryCoversTR clause
ATraffic characteristics: byte volumes, UL/DL ratio, token throughput, streaming rate, burstinessD.4.3
BResponse time and HTTP errors: end-to-end latency, TTFT, TTLT, stall events, success rate, error taxonomyD.4.4
CService metrics: agent tool calling, loop factor, tool latency, computer-use actionsD.4.5
DTask-related QoE: video understanding accuracy (clause D.4.6.1). Task completion, answer quality, satisfaction and safety are for further study.D.4.6

Open a category for its metrics.

A. Traffic characteristics (clause D.4.3)

These describe the volume and temporal shape of AI service traffic.

Volume and tokens (D.4.3.2, table D.4.3.2-1):

MetricOutput keyDefinition
Uplink bytes, meanuplink_bytes_meanMean of request_bytes over the primary records; an estimate, the size of the serialised JSON
Downlink bytes, meandownlink_bytes_meanMean of response_bytes
Total bytes, meantotal_bytes_meanSum of the two means; derived
UL/DL ratio, meanul_dl_ratio_meanMean request bytes divided by mean response bytes (ratio of means, not per record); null or infinite if the mean response is 0
Input tokens, meanuplink_tokens_meanMean of tokens in where above 0; from the API usage field, estimated with tiktoken if absent (Gemini streaming)
Output tokens, meandownlink_tokens_meanMean of tokens out where above 0
Token output rate, meantoken_output_rate_per_secTokens out divided by latency per record, then averaged, which avoids the bias of a ratio of means

Streaming throughput (D.4.3.3, table D.4.3.3-1), for chunked HTTP streaming only: streaming_byte_rate_per_sec, the mean of response bytes over the time from first to last token; and chunk_count_mean, the mean number of SSE content-delta chunks, which excludes metadata-only events.

Burstiness, application layer (D.4.3.4, table D.4.3.4-1). Burstiness is assessed on amplitude (byte volume variation) and on time (inter-request arrival). With total_bytes[i] the request plus response bytes of record i, and gaps the differences between sorted request start times:

MetricOutput keyDefinition
Burst peak-to-mean ratioburst_peak_to_mean_ratiomax(total bytes) / mean(total bytes); 1.0 for uniform traffic, above 1.0 for bursty
Burst coefficient of variationburst_coefficient_of_variationstdev(total bytes) / mean(total bytes)
Burst gap thresholdburst_gap_threshold_msburst_gap_sec times 1000; configurable, default 1000 ms
ON-gap mean and countburst_on_gap_mean_ms, burst_on_gap_countGaps at or below the threshold, within a burst
OFF-gap mean and countburst_off_gap_mean_ms, burst_off_gap_countGaps above the threshold, silence between bursts

By default the time between bursts therefore cannot be smaller than 1.0 s in the current configuration.

Burstiness, packet layer (D.4.3.4, table D.4.3.4-2). The instantaneous bit rate over a window of W ms is IBR(t, W) = bytes received in [t, t+W) times 8, divided by (W / 1000), in bit/s; the packet arrival rate PAR(t, W) is the packet count in the window; the inter-packet gap IPG is the time between consecutive arrivals in a flow.

FieldsDefinition
ibr_window_msWindow width W; recommended 1, 5, 10, 50 and 100 ms. Smaller windows show sharper peaks.
ibr_mean_bps, ibr_peak_bpsMean over windows with at least one byte; maximum over windows
ibr_peak_to_mean, ibr_cvPeak over mean; stdev over mean of the IBR series
ibr_active_fractionShare of windows with IBR above 0
ibr_p95_bps, ibr_p99_bps95th and 99th percentile of the IBR series
par_window_ms, par_mean, par_peak, par_peak_to_meanPacket-count window (recommended 1 ms); mean over non-empty windows; maximum; their ratio
ipg_mean_us, ipg_cv, ipg_p99_usMean inter-packet gap in microseconds; stdev over mean; 99th percentile
ibr_empty_ms_pct, Pcd_empty_ms_pct (as printed in the TR)Percentage of empty bins not taken into account for the IBR computation, and of empty bins for the packet density calculation

Very small windows are sensitive to clock or timestamp resolution, and a few packets in a short window can give a very high IBR. Bursts can also be measured per request type or flow, since bursts to the model server may differ from bursts to external tools. A NOTE says that burst metrics for media traffic with dynamically changing characteristics are also in TR 26.823, and that the two sets of definitions are intended to remain aligned.

Communication frequency (D.4.3.5): application-layer interactions (requests, responses, coordination or signalling messages) as communication_frequency, events per second, and inter_message_interval, the time between consecutive events. These are meant as generic metrics, also for multi-agent coordination and signalling.

Small packets (D.4.3.6): small_packet_ratio, the share of packets below a size threshold, 128 bytes by default; other thresholds may be used where justified.

B. Response time and HTTP errors (clause D.4.4)
MetricOutput keyDefinition
Response latencyresponse_latency_{mean,median,p95,p99,min,max,std}_mslatency_sec, the wall-clock time from sending the request to receiving the last byte, times 1000 (D.4.4.1)
Time to first token (TTFT)ttft_{mean,p50,p95,p99}_mst_first_token minus t_request_start, where both are present and the result is not negative (D.4.4.2)
Time to last token (TTLT)ttlt_{mean,p50,p95,p99}_mst_last_token minus t_request_start, same conditions (D.4.4.3)
Stall gap thresholdstall_gap_threshold_msstall_gap_sec times 1000; configurable, default 1000 ms (D.4.4.4)
Stall ratestall_rateStall events divided by all inter-chunk gaps, across streaming records; null if there are no streaming records
Stall events and durationstall_event_count, stall_duration_{mean,p95,p99}_msCount of gaps at or above the threshold; mean and percentiles of those gaps
Chunk countchunk_count_meanRepeated here for the streaming context
Success ratesuccess_rate_pctRecords with success true, over the primary records, times 100 (D.4.4.5)
Errorserror_count, http_errors.*Failed records classified as timeout, rate_limited_429, server_error_5xx (500 to 599, with 501 excluded as non-transient), tool_failure (failed MCP tool calls and computer-use actions) and other

TTFT is the time from the HTTP request to the first content token (or first byte, where the first bytes carry no token in some implementations). It reflects the model's initial inference latency plus the network round trip, and is the primary indicator of perceived responsiveness for streaming. For non-streaming responses TTLT equals the full latency. A stall is an inter-chunk gap at or above the threshold; the inter_chunk_times field of a streaming record holds the successive gaps in seconds.

C. Service metrics (clause D.4.5)
MetricOutput keyDefinition
Tool calls per session, meanagent.tool_calls_meanMean of the cumulative tool-call count recorded on each model turn (D.4.5.2)
Loop factoragent.loop_factorMean number of primary records per session; above 1 means multi-step reasoning, 1.0 a single turn
Tool latencyagent.tool_latency_{mean,p50,p95}_msExecution time inside the tool (tool_latency_sec), without model inference; from tool records only
Actions and steps per sessioncomputer_use.actions_per_session_mean, computer_use.steps_per_session_meanOne step is one model reasoning cycle (D.4.5.3)
Action latencycomputer_use.action_latency_{mean,p95}_msThe full screenshot, action and result cycle
Screenshot bytescomputer_use.screenshot_bytes_meanA proxy for display complexity and bandwidth cost
Action error ratecomputer_use.action_error_rateFailed actions over all computer actions, 0 to 1

The agent metrics apply where the model makes iterative MCP tool calls, each recorded separately with the type mcp_tool_call. The computer-use metrics apply only to scenarios with computer actions and do not affect latency, TTFT or byte metrics.

D. Task-related QoE (clause D.4.6)

An AI media service delivers an inference result, often text, rather than a reconstructed media signal. There is no reference signal to compare against, so reference-based media quality metrics, including those of TR 26.944, do not apply. A task-related metric is used instead, specific to each scenario because it depends on that scenario's ground truth (D.4.6.0).

Closed-form video understanding accuracy (D.4.6.1.1), for the real-time and non-real-time video understanding scenarios: A_vuc = N_correct / N_total (equation D.4.6.1-1), the share of questions answered correctly in a test session, between 0 and 1. A question is a multiple-choice question tied to a timestamp in the video, with one ground-truth option, posed at that point in the stream and answerable from the content delivered so far. A_vuc is reported per network profile with N_total, and optionally per task type; non-parseable answers count as incorrect. For values to be comparable across profiles, the corpus and subset, question set and order, evaluation context, model and inference configuration (including frame sampling rate) and the number of repetitions are held constant, and only the profile varies. With four options, random selection gives an expected A_vuc of 0.25, the lower reference point for severely impaired profiles. The metric suits visual question answering; captioning and summarisation are open-ended.

Open-ended accuracy (D.4.6.1.2) is for further study. The model's free-form answer is scored by an LLM judge on completeness and reliability, each from 0 to 5, summed to a score out of 10 (A_vuo). An Editor's Note proposes a ratio against an anchor profile, Q_vuo = A_vuo_measured / A_vuo_anchor (equation D.4.6.1-2), where 1 means unchanged answer quality. Another Editor's Note says the metric is not implemented in the testbed and no results with it are reported.

Candidate task metrics listed in D.4.6.0 as for further study are task completion rate, task abandonment rate, subtask completion rate, tool success rate, time to first decisive assertion (TTFDA), media staleness at response and validity at arrival. The same clause also says that no task-based QoE metrics are defined, although D.4.6.1 then defines video understanding accuracy.

What each traffic characteristic is measured with (clause 6.3.3.3)

TR 26.870 lists groups of traffic characteristics as relevant to other working groups, and maps each to the analysis in annex C and to these metrics (table 6.3.3.3-1).

GroupExamples from the TRAnalysisMetrics
A. Volume and directionalityUplink and downlink volume, UL/DL asymmetry, session and flow durationClause C.4Clause D.4.3 (category A)
B. Temporal structureBurstiness, inter-arrival times, periodic or event-driven, streaming or request/responseClause C.5Clause D.4.5 (category C), as printed in the TR
C. Latency sensitivityOne-way and round-trip sensitivity, time to first media element, completion latencyClauses C.5 and C.6Clause D.4.4 (category B)
D. Reliability and continuitySensitivity to loss, tolerance to reordering or jitter, stalls, retransmissionsClause C.7Clauses D.4.4 and D.4.6 (categories B and D)
E. Adaptivity and dynamicsRate adaptation, dependency on inference timing, coupling across flows or modalities, changes within a sessionClauses C.7 and C.8Clauses D.4.3 and D.4.6 (categories A and D)

The TR lists as follow-up a careful review and detailed definition of these metrics, and the definition of how the characteristics are measured. Extending the mapping to media services other than AI is for further study. The burstiness descriptors that group B relates to are defined in clause D.4.3.4 (category A); the TR table points to D.4.5.

The metrics each assumption of annex C uses (clauses C.4.3 to C.8.3) are listed on Results; some, such as multi-window burstiness, peak intra-burst rate and HTTP setup round-trip time, are marked in the TR as not yet complete in the testbed.

Sources for this page
  • At a glance and the categories: TR 26.870 V0.6.1, clauses D.4.1 and D.4.2, table D.4.1-1; annex B.1 and its Editor's Note.
  • A. Traffic characteristics: TR 26.870 V0.6.1, clauses D.4.3.1 to D.4.3.6, tables D.4.3.2-1, D.4.3.3-1, D.4.3.4-1 and D.4.3.4-2, and the NOTEs of D.4.3.4.
  • B. Response time and HTTP errors: TR 26.870 V0.6.1, clauses D.4.4.1 to D.4.4.5 and tables D.4.4.1-1 to D.4.4.5-1.
  • C. Service metrics: TR 26.870 V0.6.1, clauses D.4.5.1 to D.4.5.3, tables D.4.5.2-1 and D.4.5.3-1.
  • D. Task-related QoE: TR 26.870 V0.6.1, clauses D.4.6.0, D.4.6.1.0 to D.4.6.1.2, equations D.4.6.1-1 and D.4.6.1-2 and the Editor's Notes. Results with the metric are in annex D.5.1.
  • What each characteristic is measured with: TR 26.870 V0.6.1, clause 6.3.3.3, table 6.3.3.3-1 and its Editor's Note.