Skip to content

feat: TPP passive measurement engine (Story 20.3)#104

Merged
rajish merged 5 commits intomasterfrom
feature/story-20-3-tpp-passive-measurement
Mar 27, 2026
Merged

feat: TPP passive measurement engine (Story 20.3)#104
rajish merged 5 commits intomasterfrom
feature/story-20-3-tpp-passive-measurement

Conversation

@rajish
Copy link
Copy Markdown
Owner

@rajish rajish commented Mar 27, 2026

Summary

  • Adds PassiveTPPEngine service that correlates Claude Code log parser token data with utilization poll changes
  • Implements accumulation window with 30-minute cap, monotonic guard, and reset detection (>=50% drop)
  • Handles multi-model attribution with shared delta and confidence assignment (high/medium/low)
  • Extends TPPStorageService with query/aggregation methods (getMeasurements, getAverageTPP)
  • Integrates into PollingEngine for automatic passive measurement on each poll cycle
  • 20 new unit tests (12 PassiveTPPEngine + 8 TPPStorageService query tests)

Story

20.3: TPP Data Model and Passive Measurement Engine

Test plan

  • All XCTest unit tests pass
  • Code review findings addressed
  • Build succeeds with xcodebuild

Summary by CodeRabbit

Release Notes

  • New Features

    • Implemented passive measurement engine for tracking token usage and computing efficiency metrics
    • Added coverage health monitoring with degradation detection and automated suggestions
    • Introduced query and filtering capabilities for historical measurement data with average computations
  • Tests

    • Added comprehensive test coverage for passive measurement scenarios and data queries
  • Chores

    • Documented deferred work for future SQLite optimization

rajish added 5 commits March 28, 2026 00:32
Implement passive tokens-per-percent measurement that correlates Claude
Code log parser token data with utilization poll changes. Creates
per-model TPP measurements automatically between benchmark calibrations.

- PassiveTPPEngine with accumulation window, reset detection, monotonic
  guard, 30-min cap, multi-model attribution, and confidence assignment
- TPPStorageService extended with storePassiveResult, getMeasurements
  (filtered queries), and getAverageTPP (aggregation)
- PollingEngine integration: captures previous poll, triggers log scan,
  invokes passive engine in fire-and-forget Task
- AppDelegate wiring: log parser and TPP storage created before
  PollingEngine to enable dependency injection
- 20 test cases across PassiveTPPEngineTests and
  TPPStorageServiceQueryTests
- Extract shared INSERT logic into private insertMeasurementRecord helper
  in TPPStorageService; storeBenchmarkResult and storePassiveResult both
  delegate to it, eliminating 60-line duplication
- Move logger calls outside NSLock blocks in PassiveTPPEngine accumulation
  branches; use AccumulationAction enum to return result from lock closure
- Story 20.3 status: done; sprint-status updated
@rajish rajish merged commit 4a3d985 into master Mar 27, 2026
0 of 2 checks passed
@rajish rajish deleted the feature/story-20-3-tpp-passive-measurement branch March 27, 2026 23:32
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 27, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d44b40f3-7422-4785-89ef-2d44159a6f0d

📥 Commits

Reviewing files that changed from the base of the PR and between 49ed179 and ed23506.

📒 Files selected for processing (14)
  • _bmad-output/implementation-artifacts/20-1-active-benchmark-measurement.md
  • _bmad-output/implementation-artifacts/20-3-tpp-data-model-passive-measurement-engine.md
  • _bmad-output/implementation-artifacts/deferred-work.md
  • _bmad-output/implementation-artifacts/sprint-status.yaml
  • cc-hdrm/App/AppDelegate.swift
  • cc-hdrm/Models/PassiveTPPHealth.swift
  • cc-hdrm/Services/PassiveTPPEngine.swift
  • cc-hdrm/Services/PassiveTPPEngineProtocol.swift
  • cc-hdrm/Services/PollingEngine.swift
  • cc-hdrm/Services/TPPStorageService.swift
  • cc-hdrm/Services/TPPStorageServiceProtocol.swift
  • cc-hdrmTests/Services/BenchmarkServiceTests.swift
  • cc-hdrmTests/Services/PassiveTPPEngineTests.swift
  • cc-hdrmTests/Services/TPPStorageServiceQueryTests.swift

📝 Walkthrough

Walkthrough

This PR introduces a passive TPP measurement engine that processes consecutive usage polls to compute tokens-per-percent metrics. It adds the engine protocol/service, extends storage with query/aggregation methods, integrates into PollingEngine and AppDelegate, includes health tracking, and provides comprehensive test coverage.

Changes

Cohort / File(s) Summary
Protocol & Model Definitions
cc-hdrm/Services/PassiveTPPEngineProtocol.swift, cc-hdrm/Models/PassiveTPPHealth.swift, cc-hdrm/Services/TPPStorageServiceProtocol.swift
Defines async PassiveTPPEngineProtocol with processPoll(), getHealth(), and resetAccumulation() methods; adds PassiveTPPHealth model with coverage and degradation metrics; extends TPPStorageServiceProtocol with passive store, query, and aggregation methods.
Passive TPP Engine Implementation
cc-hdrm/Services/PassiveTPPEngine.swift
Implements PassiveTPPEngineProtocol with complex logic for computing TPP measurements from consecutive polls: handles token accumulation windows with caps/guards, detects utilization resets, computes 5-hour/7-day deltas, queries token aggregates, stores per-model measurements, tracks coverage health, and assigns confidence levels.
Storage Service Extensions
cc-hdrm/Services/TPPStorageService.swift
Refactors storeBenchmarkResult to use new private insertMeasurementRecord(_:) helper; adds storePassiveResult(_:) for passive measurements; adds getMeasurements(from:to:source:model:confidence:) with optional filtering and ascending timestamp ordering; adds getAverageTPP(from:to:model:source:) for time-range aggregation.
App Integration & Wiring
cc-hdrm/App/AppDelegate.swift, cc-hdrm/Services/PollingEngine.swift
AppDelegate now constructs and stores TPPStorageService, ClaudeCodeLogParser, and PassiveTPPEngine before PollingEngine, passing them as optional dependencies. PollingEngine stores these dependencies and, during post-persistence fire-and-forget task, conditionally triggers passive processing by scanning the log parser and calling engine's processPoll(current:previous:).
Engine Test Suite
cc-hdrmTests/Services/PassiveTPPEngineTests.swift
Comprehensive tests for PassiveTPPEngine covering basic measurement storage, accumulation flushing/capping/monotonic guards, utilization reset handling, multi-model attribution, confidence mapping, delta-only records, health reporting, and 7-day delta computation.
Storage Query Tests
cc-hdrmTests/Services/TPPStorageServiceQueryTests.swift, cc-hdrmTests/Services/BenchmarkServiceTests.swift
New test file exercises storage query/aggregation with measurement insertion, filtering, timestamp ordering, and average computation; mock in BenchmarkServiceTests extended to support new storage protocol methods.
Documentation & Status
_bmad-output/implementation-artifacts/20-1-active-benchmark-measurement.md, _bmad-output/implementation-artifacts/20-3-tpp-data-model-passive-measurement-engine.md, _bmad-output/implementation-artifacts/deferred-work.md, _bmad-output/implementation-artifacts/sprint-status.yaml
Updated story 20-1 status to ready-for-dev with checklists reset; added complete 20-3 story artifact documenting passive TPP engine implementation; added deferred-work file noting Int32 truncation concern in SQLite bindings; updated sprint status markers for 20-2 and 20-3.

Sequence Diagram(s)

sequenceDiagram
    participant PollingEngine
    participant HistoricalDataService
    participant PassiveTPPEngine
    participant ClaudeCodeLogParser
    participant TPPStorageService

    PollingEngine->>HistoricalDataService: getLastPoll()
    HistoricalDataService-->>PollingEngine: previous UsagePoll
    
    PollingEngine->>PassiveTPPEngine: processPoll(current, previous)
    
    PassiveTPPEngine->>ClaudeCodeLogParser: scan()
    ClaudeCodeLogParser-->>PassiveTPPEngine: token aggregates
    
    PassiveTPPEngine->>PassiveTPPEngine: compute deltas, validate resets
    PassiveTPPEngine->>PassiveTPPEngine: accumulate or flush window
    
    PassiveTPPEngine->>TPPStorageService: storePassiveResult(measurement)
    TPPStorageService-->>PassiveTPPEngine: success/failure
    
    PassiveTPPEngine-->>PollingEngine: PassiveTPPHealth
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Poem

🐰 Hop, hop! The polls spin round and round,
Tokens hidden till they're found.
Passive engines now can peek—
Seven days and five per week!
Health and coverage, oh what cheer,
Measurements appear!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/story-20-3-tpp-passive-measurement

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant