Skip to content

Improve docs: quickstart guide, standardize README, add results table #3

@rajkumar42

Description

@rajkumar42

Summary

Make this benchmark repo more accessible to new contributors by adding a quickstart guide, standardizing the README structure, and including a results summary table.

What needs to happen

Quickstart ("Run your first benchmark in 5 minutes")

  • Add a short section at the top of README with minimal steps to run the benchmark
  • Include: clone, install deps, set API key, run a small subset
  • Example: python run_benchmark.py --limit 5 to run just 5 tasks

Standardize README

  • Follow a consistent structure: Overview → Quickstart → Full Setup → Results → Contributing
  • Align with other OpenSymbolicAI benchmark repos for consistency
  • Include badges (Python version, license, etc.)

Results summary table

  • Add a markdown table showing results per model/provider
  • Columns: Model | Pass Rate | Avg Tokens | Avg Cost | Date
  • Keep the table updated as new provider results come in

Acceptance criteria

  • A newcomer can go from clone to first benchmark result in under 5 minutes
  • README follows the standardized structure
  • Results table is present with at least one row of data

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions