Skip to content

Make optional deps safe, fix imports, harden llama-cli and runner registry#4

Merged
harvatechs merged 1 commit intomainfrom
codex/fix-bugs-and-prepare-for-production
Feb 10, 2026
Merged

Make optional deps safe, fix imports, harden llama-cli and runner registry#4
harvatechs merged 1 commit intomainfrom
codex/fix-bugs-and-prepare-for-production

Conversation

@harvatechs
Copy link
Owner

Motivation

  • Prevent runtime/test failures when optional native deps (llama_cpp, torch) are not installed by treating them as optional and failing gracefully.
  • Improve robustness of the FastAPI runner so model registry loading is independent of process CWD while still honoring ARIV_MODELS_YAML.
  • Surface useful errors for llama.cpp CLI failures and avoid silent subprocess failures during streaming.

Description

  • Treat llama_cpp and torch as optional runtime dependencies by guarding imports and branching behaviour when they are absent, and add an explicit ModelLoadError when attempting to load models without llama_cpp.
  • Replace fragile parent-relative imports with absolute imports for the tools/config modules in core modules to avoid import-time crashes.
  • Harden LlamaCLI.stream_chat to asynchronously collect stderr and raise a detailed RuntimeError (binary, model, exit code, stderr tail) if the subprocess exits with a non-zero status.
  • Make runner registry path resolution independent of the current working directory by resolving ARIV_MODELS_YAML or falling back to a path relative to the package file.
  • Update and add tests to reflect API/behavioural adjustments (VRAM manager API alignment, optional-torch handling, runner registry cwd-independence, and llama-cli non-zero-exit handling).

Testing

  • Ran the full test suite with pytest -q and verified success: 30 passed, 2 skipped.
  • Executed focused test runs including tests/test_runner_app.py, tests/test_llama_cli.py, tests/test_vram.py, tests/test_pipeline.py, and tests/test_orchestrator.py to validate changes, all of which passed in targeted runs.
  • Skipped tests are CUDA-dependent and will be skipped automatically when CUDA is not available in the environment.

Codex Task

@harvatechs harvatechs merged commit 0fc815f into main Feb 10, 2026
0 of 2 checks passed
@harvatechs harvatechs deleted the codex/fix-bugs-and-prepare-for-production branch February 11, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant