How we measure model speed on Borg

Borg model pages report gateway observations for first response and end-to-end output speed. The figures come from SearchAtlas AI Gateway production request logs, not vendor benchmark runs, and do not predict every future request.

First response is a proxy

The telemetry note calls the first metric time to first token, but defines it as the time until the first streamed response event reaches the gateway. This is the closest available measure, not a verified timestamp for the first visible word on your screen.

A streamed event is not necessarily the first visible token.

Model pages show the median and 95th percentile in milliseconds, along with the number of timed samples. The median is the middle observation; the 95th percentile shows a slower part of the observed distribution. Neither is a deadline.

Output speed includes the whole turn

End-to-end output tokens per second divides provider-reported output tokens by the whole turn’s elapsed time. The duration includes queueing and reasoning. It is not a streaming per-token rate.

A model that pauses before output can have a lower end-to-end rate even if its visible output later arrives quickly. These two timing rows describe different parts of the experience.

End-to-end speed includes the whole turn.

The gateway data has fields for streaming generation speed, but the recorded generation sample count is zero for the entries checked. Borg does not publish a streaming tokens-per-second figure without samples. Request tasks and output lengths also vary, so comparing medians as a controlled head-to-head benchmark would overstate what the logs show.

Sample counts and missing figures

A metric needs enough observations of its own to be shown. A high overall request count does not replace enough timed samples. Each row carries its own sample count because first-response timing and end-to-end throughput may come from different requests.

A blank is not a zero.

If a model has insufficient timed observations, the page says there is not enough data to publish a speed figure. Non-text output models are excluded: text tokens per second would not describe producing image, video, or audio output.

Success is a separate measure

Where shown, request success rate is the share of requests Borg completed successfully, excluding requests canceled by the client. Borg publishes the rate only at 99% or above. Below that threshold, telemetry records a note instead of a public percentage.

That absence is not a claim of perfect reliability or a published estimate of failures. Success has its own sample count and a different denominator from timing rows.

Read each figure with its sample count and notes.

Use the models directory to find a model page. Speed in these tables is an observation of gateway traffic, with known limits, not a promise that a future request will match the median.