-
Notifications
You must be signed in to change notification settings - Fork 0
Improve docs: quickstart guide, standardize README, add results table #3
Copy link
Copy link
Open
Labels
good first issueGood for newcomersGood for newcomers
Description
Summary
Make this benchmark repo more accessible to new contributors by adding a quickstart guide, standardizing the README structure, and including a results summary table.
What needs to happen
Quickstart ("Run your first benchmark in 5 minutes")
- Add a short section at the top of README with minimal steps to run the benchmark
- Include: clone, install deps, set API key, run a small subset
- Example:
python run_benchmark.py --limit 5to run just 5 tasks
Standardize README
- Follow a consistent structure: Overview → Quickstart → Full Setup → Results → Contributing
- Align with other OpenSymbolicAI benchmark repos for consistency
- Include badges (Python version, license, etc.)
Results summary table
- Add a markdown table showing results per model/provider
- Columns: Model | Pass Rate | Avg Tokens | Avg Cost | Date
- Keep the table updated as new provider results come in
Acceptance criteria
- A newcomer can go from clone to first benchmark result in under 5 minutes
- README follows the standardized structure
- Results table is present with at least one row of data
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
good first issueGood for newcomersGood for newcomers