Add AbsoluteCaptureTime extension and related functionality

[lmiguelgato]

Add support for abs-capture-time RTP header extension

Summary

Implements the abs-capture-time RTP header extension as specified in the WebRTC specification. This extension enables accurate audio/video synchronization in scenarios where media passes through RTCP-terminating mixers or other intermediary systems.

Motivation

The abs-capture-time extension addresses a critical limitation in multi-hop WebRTC scenarios. When RTP packets traverse mixers or SFUs that terminate RTCP (Sender Reports), receivers lose the ability to correlate timestamps from the original capture system, breaking A/V sync. This extension embeds the original capture timestamp in each RTP packet, preserving synchronization information end-to-end.

Key use cases:

• Multi-point conferencing through mixing servers
• Cloud recording services that need accurate A/V sync
• Cascaded SFU architectures
• Any scenario with RTCP-terminating intermediaries

Implementation Details

Extension Format

• URI: http://www.webrtc.org/experiments/rtp-hdrext/abs-capture-time
• Short format (8 bytes): 64-bit NTP timestamp (UQ32.32 fixed-point)
• Extended format (16 bytes): Timestamp + 64-bit signed clock offset

Changes

1. New Extension Variant (src/rtp/ext.rs:27)

   • Added Extension::AbsoluteCaptureTime enum variant

2. Extension Values (src/rtp/ext.rs:866-869)

   • abs_capture_time: Option<SystemTime> - Absolute NTP timestamp of original frame capture
   • abs_capture_clock_offset: Option<i64> - Optional sender's clock offset estimate (raw NTP Q32.32 format)

3. Serialization/Parsing (src/rtp/ext.rs)

   • Converts between SystemTime and 64-bit NTP format (UQ32.32) using as_ntp_64()/from_ntp_64()
   • Handles variable-length format (8 or 16 bytes)
   • Uses two-byte header form only when extension ID > 14 (16-byte payload fits in one-byte form)
   • Follows the exact pattern of SenderInfo in src/rtp/rtcp/sr.rs

4. Media Type Support

   • Enabled for both audio and video streams
   • Follows same patterns as existing abs-send-time extension

Design Decision: SystemTime vs Instant

This implementation uses SystemTime for abs_capture_time rather than Instant:

• Why SystemTime: The abs-capture-time extension represents an absolute NTP wall-clock timestamp that must be preserved across machines and network hops. Instant is a monotonic clock relative to an arbitrary local epoch and cannot represent absolute timestamps from remote systems.

• Precedent: This follows the exact pattern of SenderInfo.ntp_time in src/rtp/rtcp/sr.rs, which also uses SystemTime for NTP timestamps.

• Simplification: Using SystemTime enables direct NTP conversions (as_ntp_64()/from_ntp_64()) without complex epoch calculations.

• Note: This differs from abs-send-time, which uses Instant because it's a 24-bit truncated value used only for relative timing/bandwidth estimation, not as an absolute timestamp.

Testing

• 3 new unit tests covering all format variations:
  • abs_capture_time_short_form - 8-byte format (timestamp only)
  • abs_capture_time_extended_form - 16-byte format with clock offset
  • abs_capture_time_two_byte_form - Two-byte header form (extension ID > 14)
• 2 new integration tests (tests/abs-capture-time.rs):
  • abs_capture_time_negotiation - Verifies SDP negotiation and packet delivery
  • abs_capture_time_sdp_roundtrip - Verifies extension URI in SDP
• All existing tests pass (527 library tests)
• Timestamp accuracy verified within 1ms through roundtrip tests
• No breaking changes to existing APIs

Event Size Impact

The addition of two Option fields to ExtensionValues increased the Event enum size by 2 bytes (470 → 472 bytes). Updated the size limit to 490 bytes to provide reasonable headroom for future additions.

Usage Example

use str0m::rtp::{Extension, ExtensionMap, ExtensionValues};
use std::time::SystemTime;

// Configure extension mapping (typically done during SDP negotiation)
let mut exts = ExtensionMap::standard();
exts.set(9, Extension::AbsoluteCaptureTime);

// Set capture time when sending RTP packets
let mut ext_vals = ExtensionValues::default();
ext_vals.abs_capture_time = Some(SystemTime::now());

// For mixers/SFUs: optionally include clock offset (raw NTP Q32.32 format)
ext_vals.abs_capture_clock_offset = Some(estimated_offset_ntp);

// Extension is automatically serialized into RTP packets
// and parsed on the receiving end

Bandwidth Considerations

Per the specification, implementations should not send this extension with every packet to save bandwidth. Senders should include it:

• Periodically (e.g., every second)
• When the capture system changes
• At the start of a stream

This implementation provides the mechanism; rate limiting policy is left to the application layer.

Compatibility

• No breaking changes - purely additive
• Applications not using this extension are unaffected
• Follows existing extension architecture patterns
• Compatible with Chrome, Firefox, and other WebRTC implementations

References

• abs-capture-time specification
• RFC 8285 - RTP Header Extensions
• Related extension: abs-send-time (already implemented in str0m)

[xnorpx]

@lmiguelgato also add an integration test (in the test folder)

[algesten]

Overall

Good, well-structured addition that follows the existing extension patterns closely. The serialization/deserialization logic is correct and the tests cover the key format variations. A few items to address:

Issues

1. requires_two_byte_form is unnecessarily conservative — Per RFC 8285, one-byte form supports payloads up to 16 bytes. The 16-byte extended format fits, so forcing two-byte form when abs_capture_clock_offset is present wastes header space. The self-contradicting comment confirms the confusion. Either remove the override or document a specific interop reason.

2. Remove leftover eprintln! in src/lib.rs — debug print in event_is_reasonably_sized test.

3. abs_capture_clock_offset is an opaque i64 with no documented format — The spec defines this as a signed NTP fixed-point number (SQ32.32). Without documentation, callers will misinterpret the unit. At minimum add a doc comment; ideally provide a conversion helper.

4. Potential Instant underflow in parse path — If an incoming NTP timestamp maps to a time before BEGINNING_OF_TIME, the subtraction already_happened() + duration_since_epoch - epoch_to_beginning() can panic. Consider a checked_sub/saturating approach for robustness against malformed packets.

Nits

• Byte parsing can use buf[..8].try_into().unwrap() instead of listing all 8 indices.

Observations (non-blocking)

• The extension is not added to ExtensionMap::standard(), meaning it won't be negotiated unless the application explicitly configures it. This seems intentional and fine for an experimental extension, just worth calling out.
• The event size bump from < 470 to < 490 gives ~18 bytes of headroom, which is reasonable.

[algesten]

Fundamental type issue: Instant vs SystemTime for abs_capture_time

The most significant concern with this PR is that abs_capture_time is typed as Option<Instant>, but it should be Option<SystemTime>.

Why Instant is wrong here

Instant is a monotonic clock relative to an arbitrary local epoch (typically process start or system boot). It cannot represent an absolute wall-clock time that originated on a remote system. The abs-capture-time extension exists specifically to carry the original capture timestamp across RTCP-terminating intermediaries — it's an absolute NTP wall-clock time by definition.

With Instant:

• The value is meaningless across machines (different epochs)
• Round-tripping through Instant → SystemTime → NTP → SystemTime → Instant introduces unnecessary conversion error and complexity (as seen in the parse path)
• A mixer/SFU forwarding this value would need to reconstruct it relative to its own Instant epoch, defeating the purpose
• The field cannot be meaningfully set by an application that receives a capture timestamp from an external source (e.g., a hardware encoder providing a wall-clock time)

Why SystemTime is correct

SystemTime maps directly to NTP timestamps. The conversion is straightforward (SystemTimeExt::as_ntp_64 / SystemTime::from_ntp_64), already used elsewhere in str0m (SR reports, XR blocks). There's no need for the already_happened() + duration - epoch_to_beginning() gymnastics in the parse path.

Why abs_send_time using Instant is different

The existing abs_send_time uses Instant but that's a fundamentally different use case:

• It's a 24-bit truncated value that wraps every ~64 seconds
• It's used for relative timing (bandwidth estimation, REMB)
• It's rebased on receipt relative to local time anyway (update_absolute_send_time)
• It's not meant to be forwarded as an absolute timestamp

abs-capture-time has none of these properties — it's a full 64-bit NTP timestamp meant to be preserved end-to-end.

Suggested change

pub abs_capture_time: Option<SystemTime>,

This simplifies both the write and parse paths and makes the API correct by construction.

[algesten]

To clarify the SystemTime point above — SenderInfo in src/rtp/rtcp/sr.rs is the exact precedent to follow. It has the same wire format (64-bit NTP timestamp) and uses SystemTime directly:

// Field (sr.rs:29)
pub ntp_time: SystemTime,

// Write (sr.rs:75-76)
let mt = self.ntp_time.as_ntp_64();
buf[4..12].copy_from_slice(&mt.to_be_bytes());

// Parse (sr.rs:123-126)
let ntp_time = u64::from_be_bytes([buf[4], buf[5], buf[6], buf[7], buf[8], buf[9], buf[10], buf[11]]);
let ntp_time = SystemTime::from_ntp_64(ntp_time);

The abs_capture_time field should follow this same pattern — Option<SystemTime> with as_ntp_64() / from_ntp_64(). This would eliminate the already_happened() + duration_since_epoch - epoch_to_beginning() conversion and the potential Instant underflow issue.

[algesten]

If you want to discuss stuff, you can reach me on Discord also.

[lmiguelgato]

@lmiguelgato also add an integration test (in the test folder)

Added test case abs_capture_time_negotiation with MediaKind::Audio, where it's verified that the SDP string of the offer contains abs-capture-time extension, and that packets are flowing and received correctly.

Also added test case abs_capture_time_sdp_roundtrip with MediaKind::Video, where it's checked that both offer and answer contain abs-capture-time.

[lmiguelgato]

To clarify the SystemTime point above — SenderInfo in src/rtp/rtcp/sr.rs is the exact precedent to follow. It has the same wire format (64-bit NTP timestamp) and uses SystemTime directly:

// Field (sr.rs:29)
pub ntp_time: SystemTime,

// Write (sr.rs:75-76)
let mt = self.ntp_time.as_ntp_64();
buf[4..12].copy_from_slice(&mt.to_be_bytes());

// Parse (sr.rs:123-126)
let ntp_time = u64::from_be_bytes([buf[4], buf[5], buf[6], buf[7], buf[8], buf[9], buf[10], buf[11]]);
let ntp_time = SystemTime::from_ntp_64(ntp_time);

The abs_capture_time field should follow this same pattern — Option<SystemTime> with as_ntp_64() / from_ntp_64(). This would eliminate the already_happened() + duration_since_epoch - epoch_to_beginning() conversion and the potential Instant underflow issue.

This makes total sense. Thanks for bringing it up. I have changed it to SystemTime, and now it's also not needed to do the underflow check to avoid panic with bogus timestamps. Thanks.

[lmiguelgato]

Rebased and squashed my changes after addressing the code review comments. Updated PR description.

[xnorpx]

change lgtm

[xnorpx]

@algesten are you good with the changes?

[algesten]

Made a tweak AbsCaptureTime to encapsulate these two values since they don't exist independently.

[xnorpx]

@algesten changelog entry?

[algesten]

Thanks!