Skip to content

Cecli Atomic Tests#312

Merged
dwash96 merged 49 commits intocecli-dev:v0.91.0from
ErichBSchulz:cat
Dec 24, 2025
Merged

Cecli Atomic Tests#312
dwash96 merged 49 commits intocecli-dev:v0.91.0from
ErichBSchulz:cat

Conversation

@ErichBSchulz
Copy link
Copy Markdown

@ErichBSchulz ErichBSchulz commented Dec 24, 2025

See the README for the vision.

Known issues:

  • JS can CPP tests currently broken

Unknown issues:

  • probably lots

dwash96 and others added 30 commits December 21, 2025 15:53
@dwash96
Thinking... to Processing... for agnosticism
5e07364
@ErichBSchulz
tweak readme
c06989a
@ErichBSchulz
tweaks
ea14ba7
@ErichBSchulz
begin cleanup
22fe4ab
@ErichBSchulz
feat: Add --dry mode to skip docker check and tests
06b5b04 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
chore: Run linter and format code
1738021 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
feat: Add dry run option to benchmark
5eaf450 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
chore: Run linter on benchmark files
1b0d525 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
feat: Replace print with logging and add verbose/quiet flags
c685caf 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
chore: Run linter
c9d0f06 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
fix: Change default benchmark exercises directory
a8e8a1b
@ErichBSchulz
refactor: Add logging and comments to resolve_dirname
0121aeb 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
feat: Rename dirname to results_dir for clarity
c70e766
@ErichBSchulz
fix: Replace asserts with explicit error logging and exit
8430e02 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Resolve linter errors in benchmark script
601e8c3 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
hacking
bd5afe2
@ErichBSchulz
feat: Add support for new cat exercise structure
85e1556 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Run linter and fix formatting issues
14cb852 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
chore: Add logging for found exercises and metadata
7df0b0f
@ErichBSchulz
fix: Import importlib_resources at the top level
f24d56d 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Move models import to top level in benchmark script
b021795 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
refactor: Dry out run_test code for single and multi-threaded execution
a3dc824 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Correct indentation in benchmark script
3cc0011 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Set commit hash to '???????' when dry run
f50685c
@ErichBSchulz
docs: Update --hash-re help text for fractional set division
1969da4 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Update placeholder commit hash in benchmark script
02164b6 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
docs: Update hash-re help text with nth char and hex range examples
9fe5f24 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
docs: Add enhancements section to benchmark README
3f25430 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
chore: Update benchmark README with linting fixes
a1c011f 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Add debug logging for metadata parsing
c9b13bb
@ErichBSchulz
refactor: Support new dir structure and dry run
ea4df0f 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
chore: Fix linter warnings in benchmark.py
c29a88e 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
docs: Add debug plan for benchmark script
2190623 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
chore: Update benchmark debug plan for async/await
c7f89c9 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
feat: Update benchmark script for async Coder
8390a20 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
fix: Handle None results_dir in benchmark script
e7e56f0 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
fix: Add helpful error message with paths to resolve_dirname
86a2b20 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
fix: Add debug logging for directory resolution and result loading
c79c7bf 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
fix: Exit when prior unnamed runs exist
28dfc22 
Co-authored-by: aider-ce (gemini/gemini-3-pro-preview)
@ErichBSchulz
refactor: Simplify benchmark result directory structure
fa13a81 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Improve readability of language filtering in benchmark
b568600 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
change to gem
54442fe
@ErichBSchulz
refactor: Copy new exercises when continuing benchmark runs
8150fef 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Format benchmark.py to conform to linter
29e3c5d 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
refactor: Update benchmark README with Cecli Cats details
9c2359e 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
fix: Update benchmark README with minor formatting changes
663cba4 
Co-authored-by: aider-ce (gemini/gemini-3-flash-preview)
@ErichBSchulz
yada
65bdb1d
@ErichBSchulz
Merge remote-tracking branch 'dwash/main' into cat
68c00af
@ErichBSchulz
yada
fde8469
@dwash96 dwash96 changed the base branch from main to v0.91.0 December 24, 2025 05:10
@dwash96 dwash96 merged commit 062de57 into cecli-dev:v0.91.0 Dec 24, 2025
7 of 8 checks passed
@dwash96 dwash96 mentioned this pull request Dec 24, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ErichBSchulz @dwash96