Skip to content

benchmark: add semantic and hybrid mode benchmark runs #49

@salishforge

Description

@salishforge

Current State

Only keyword mode has been benchmarked (88.0% R@5). Semantic and hybrid modes require an embedding provider.

Action

Run the benchmark with:

  • `BENCHMARK_MODES=semantic EMBEDDING_PROVIDER=ollama`
  • `BENCHMARK_MODES=hybrid EMBEDDING_PROVIDER=ollama`

Compare against keyword baseline. Document in RESULTS.md.

Expected Improvement Areas

  • single-session-preference (implicit language → semantic similarity)
  • temporal-reasoning (date relationships → embedding proximity)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttestingTest coverage and quality

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions