Skip to content

Add non-record streaming legal TTT late-block submission#662

Open
simon-marcus wants to merge 3 commits intoopenai:mainfrom
simon-marcus:non-record-streaming-legal-ttt
Open

Add non-record streaming legal TTT late-block submission#662
simon-marcus wants to merge 3 commits intoopenai:mainfrom
simon-marcus:non-record-streaming-legal-ttt

Conversation

@simon-marcus
Copy link

Summary

This adds a non-record track_10min_16mb submission documenting a full 8xH100 run built from the March 23 leader with a different legal TTT protocol.

Why submit this

This is intentionally framed as a non-record submission. The main value is to provide the community with a clean comparison between:

  • the March 23 leader's chunked score-first legal TTT over all blocks
  • a streaming legal TTT variant that adapts online and only updates the last 4 blocks

What changed relative to the March 23 leader

  • kept the same base model stack and LeakyReLU(0.5)^2 activation
  • switched eval-time adaptation to TTT_MODE=stream
  • used TTT_PARAM_MODE=late_blocks with TTT_LAST_N_BLOCKS=4
  • included explicit preflight and run logs for reproducibility

Result

Final legal_ttt_exact on the included run:

  • 1.12082320

This slightly trails the March 23 leader's seed-1337 1.11922988, but the intermediate pre-TTT and sliding-window metrics are very close, which makes this a useful data point on the TTT protocol itself.

Included artifacts

  • train_gpt.py
  • train_seed1337.log
  • runpod_launch.log
  • preflight.log
  • submission.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant