flowchart TD
accTitle: Object delivery timeout example
accDescr: The relay receives the complete object header at zero milliseconds. At 120 milliseconds it checks whether to send the object. The 100 millisecond limit has passed, so it resets a subgroup stream or drops a datagram.
A["0 ms: header received"] --> B["120 ms: ready to send"]
B --> C["Age exceeds 100 ms"]
C --> D["Subgroup: reset stream"]
C --> E["Datagram: drop it"]
Chapter 06
Latency and Congestion
On this page
A video can play smoothly and still be seconds behind the person speaking. For someone watching a lecture, that delay might be acceptable. For someone asking the lecturer a question, it makes the conversation difficult.
Keeping playback close to live takes more than a fast connection. The encoder has to produce media at a rate the network can carry, and the player has to decide how long to wait for missing or delayed content. MoQ gives the sender ways to choose which content to deliver and when to give up on it.
Latency
Playback latency is the time between capturing media and presenting it to the viewer. Every stage adds to it: capture, encoding, transmission, buffering, decoding, and display. Data can also wait in queues at the publisher, relay, or receiver.
Network delay is only part of this total. A frame that crosses the network quickly can still spend a long time waiting in the player. Likewise, removing a relay hop will do little for a delay caused mainly by encoding or buffering.
Round-trip time, or RTT, measures how long it takes to send data and receive a response. QUIC uses acknowledgment timing to estimate it. A short RTT helps the sender recover from loss sooner, but it doesn’t tell us how far playback is behind capture. See RFC 9002 §5.
Jitter and buffering
Frames don’t always arrive at evenly spaced intervals. Some take longer than others, and several may arrive together after a delay. This variation in arrival timing is jitter.
The player buffers media so it can present frames at their intended times despite uneven arrivals. More buffered media gives a late frame more time to arrive, but also makes the viewer wait longer. With a smaller buffer, playback stays closer to live but is more likely to run out of media.
There’s no single buffer size that suits every application. A lecture broadcast can tolerate more delay than a conversation with the lecturer. The player has to choose how much delay is acceptable and what to do when a frame misses its playback time.
Bandwidth
The network must carry media as quickly as the publisher produces it. We describe the media’s bitrate in bits per second. For example, 4 Mb/s means four million bits each second, or half a megabyte before packet headers and other overhead.
The capacity available to a connection changes as it shares links with other traffic. A video that fits comfortably at the start of a session may become too large when another device starts a download or the viewer moves to a weaker wireless connection.
Average bitrate also hides bursts. A large keyframe takes more time to send than a small frame, even when the average fits within the available capacity. A queue can absorb a short burst. It cannot absorb a sustained excess without growing.
Congestion
When more traffic reaches a network link than it can forward, packets wait in a queue. As the queue grows, packets take longer to arrive. If the queue fills, packets may be dropped.
Congestion control adjusts the transport’s sending behavior in response to network feedback. QUIC uses acknowledgments to track delivery and detect loss; a congestion controller limits how much data can be in flight. The controller described in RFC 9002 §7 reduces its congestion window when it detects loss or explicit congestion signals. QUIC implementations can use other controllers.
Slowing transmission helps the network, but the publisher may keep producing media at the same rate. That moves the waiting data into a queue at the sender. Unless the application also changes what it sends, playback can keep falling behind.
Datagrams can wait too
Choosing datagrams doesn’t remove network queues. A router can hold a packet for a long time before forwarding it, even if nobody will retransmit it after loss. Excessive queueing like this is called bufferbloat. The viewer can receive most of the media and still be far behind.
QUIC datagrams also share the connection’s congestion controller. When it prevents transmission, the sender must wait or drop the datagram. Skipping retransmission changes how loss is handled; it doesn’t guarantee prompt delivery. See RFC 9221 §5.4.
The moq.dev article Never* use Datagrams explains why timely delivery takes more than allowing packet loss. It argues for using independent streams, prioritizing useful content, and cancelling work that has become too old. Those choices still need congestion control to keep queues from growing.
Flow control
A receiver can also be too slow to consume incoming data. QUIC’s flow control lets it limit how much stream data the sender may send, both per stream and across the connection. This protects the receiver’s buffers. It is separate from congestion control, which responds to conditions on the network path. See RFC 9000 §4.
Neither mechanism knows whether the viewer still wants an old video frame. That decision belongs to the media application.
Media adaptation
To keep up with a slower connection, the application must reduce the amount of media it sends. The publisher can lower its encoding bitrate, or the viewer can request a lower-bitrate rendition. This may mean a smaller picture, fewer frames, or more compression.
Suppose a publisher produces 4 Mb/s while a viewer’s connection carries only 2 Mb/s. Ignoring overhead, each second adds 2 Mb to the sender’s queue. After five seconds, 10 Mb are waiting.
Switching to a 2 Mb/s rendition stops the queue from growing, but the old data is still there. Switching to 1 Mb/s leaves another 1 Mb/s available to clear the backlog, which would take ten seconds. These rates are simplified, but the consequence matters: reducing bitrate doesn’t immediately remove accumulated delay.
If that wait is too long, the sender can abandon old content and deliver a newer decoding starting point. It still needs to reduce the bitrate enough to prevent the queue from filling again.
Priorities
Suppose Alice’s connection can no longer keep up with both audio and video. Sending audio first may let her keep following the lecture while the picture struggles. The scheduler chooses which pending data to send, so it needs to know that audio matters more here.
MOQT expresses these choices through priorities and group ordering. In draft-21, smaller priority numbers mean more urgent work. Its recommended scheduler checks subscriber priority first, then publisher priority. When priorities are equal within a subscription, group ordering helps decide which group goes first. Implementations can use other scheduling algorithms. See §5.1 for the full recommendation.
If we send audio first and it uses all the available capacity, video still has to wait. Priority helps us choose who waits; a timeout lets us decide when waiting has gone on too long. When using a library, check its priority conventions, since moq-lite and MOQT use different rules.
Abandoning old content
Suppose video group 41 remains incomplete while group 42 provides a fresh decoding starting point. Finishing group 41 would preserve older pictures, but sending group 42 may bring Alice closer to the lecturer. Independent groups make that choice possible without requiring every preceding picture.
The sender still has to respect the stream’s structure. A reliable byte stream cannot deliver later bytes past a missing section. To abandon a subgroup’s remaining stream data, MOQT resets that stream. A receiver must then use the media format’s dependencies to decide where decoding can resume.
Draft-21 has two optional timeouts for this. One checks how long an object has waited before entering the transport. The other limits the time to finish delivering a completed subgroup. The examples below use a relay’s clock and a 100 ms limit. See §5.2.
Waiting to enter the transport
The object delivery timeout measures age from when the last byte of an object’s header arrives from upstream. At the original publisher, it starts when the application supplies that header. Before handing the object to the underlying transport, the implementation checks its age.
Here the object has already spent too long waiting before transmission. For a subgroup, expiry resets the stream rather than removing a few bytes from its middle. For a datagram, the expired message is dropped.
This check is most effective when the application can still control the queue. If it hands large amounts of data to the transport in advance, expired bytes may already be beyond its reach. Draft-21 therefore recommends minimizing transport buffering and checking expiry before retransmission where the transport interface permits it.
Waiting for delivery to finish
The subgroup delivery timeout starts when the endpoint knows all objects in the subgroup have been published. At a relay, receiving the upstream stream’s FIN provides that information. This timer covers remaining delivery work, including recovery after packet loss.
flowchart TD
accTitle: Subgroup delivery timeout example
accDescr: At 50 milliseconds the relay learns the subgroup is complete and starts a 100 millisecond timer. At 150 milliseconds it resets the stream if transport delivery is still incomplete.
A["50 ms: subgroup complete"] --> B["Start 100 ms timer"]
B --> C["150 ms: still undelivered?"]
C -->|Yes| D["Reset stream"]
C -->|No| E["Delivery finished"]
The transport must reach what the specification calls “all data committed” before the timer expires: its completion state for delivering the data, beyond merely accepting bytes into a send buffer. This concerns transport delivery, not whether the player has decoded or displayed the media. Otherwise, the stream is reset. Unlike the object age check, this timer can expire after the sender has handed all the data to the transport.
For datagrams, the subgroup timeout instead uses the same age check as the object timeout; if both are nonzero, the smaller value applies. Zero disables the corresponding timeout. These rules describe a single hop’s delivery decisions. They do not create a capture-to-display deadline across the whole relay network.
Maximum age in moq-lite
The moq.dev moq-lite guide describes a maximum age relative to the media timeline. Older groups can become unwanted as newer media appears. A network stall by itself does not advance that timeline, and sending a backlog in a burst does not make its media new.
Both publisher and subscriber apply the limit. This matters when viewers share upstream delivery: content still useful to a patient viewer may already be too old for someone speaking in the call. Keep this implementation setting separate from draft-21’s local delivery timers and from a player’s capture-to-display delay.
Returning to live playback
After the network recovers, Alice may still be watching frames that reached her earlier. A fresh transport queue does not empty the player’s playback queue. The player needs its own policy for skipping stale decoded frames or moving its presentation clock forward, while respecting audio synchronization and decoding dependencies.
Record when fresh objects arrive and when their content is displayed. If arrival recovers but display stays behind, inspect the player’s queue.
Try briefly reducing available bandwidth, then restoring it. Watch whether playback returns near live, stays behind, or repeatedly changes rendition. Record interruptions as well as average delay. A large keyframe can briefly exceed the path’s capacity even when the average bitrate fits, so the sequence of events matters.
Use a repeatable recorded source for the first experiment. It keeps scene complexity and encoder load stable while you investigate the network response. Once you understand that run, repeat on real devices and networks, where capture, rendering, and additional queues can change the result.