Skip to content

Retroactive generalizability scoring for existing memories #360

@toejough

Description

@toejough

Context

With generalizability gating implemented, new memories will have generalizability (1-5) and project_slug fields. But 3,140+ existing memories have neither.

Problem

Existing memories include significant noise — project-specific debugging observations, session status updates, and narrow context that would have been filtered by the new generalizability gate. These continue to be surfaced and consume context window budget.

Proposed approach

Batch-score existing memories:

  1. Read each memory's title, content, principle, and keywords
  2. Pass to Haiku with the same generalizability scale (1-5) used at extraction time
  3. Write the score back to the TOML file
  4. Optionally: archive or remove memories scoring < 2 (with user confirmation)

For project_slug, this is harder — the originating project isn't recorded. Options:

  • Leave blank for existing memories (they predate project tracking)
  • Attempt to infer from keywords/content (fragile, probably not worth it)

Considerations

  • This is a one-time migration, not an ongoing process
  • Could be implemented as an engram migrate subcommand
  • Should be dry-run capable (show what would be scored/removed without doing it)
  • Cost: ~3,140 Haiku calls (cheap, but consider batching)

Related

  • Spec: docs/superpowers/specs/2026-03-22-generalizability-gating-design.md (section: "What this does NOT cover")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions