Skip to content

refactor: rewrite workflow bash scripts in Python #119

@aireilly

Description

@aireilly

Summary

The three git workflow scripts (create_mr.sh, commit.sh, prepare_branch.sh) are hybrid bash/Python — they shell out to python3 -c for all JSON handling, URL normalization, and output formatting, while using bash for git commands and argument parsing. This split causes real bugs (see #118), limits testability, and increases maintenance burden. This issue proposes rewriting them as pure Python modules.

Evidence: current language split

Script Total lines Inline Python lines python3 invocations Python %
create_mr.sh 378 64 11 17%
commit.sh 209 34 4 16%
prepare_branch.sh 159 11 1 7%

For context, 6 standalone Python scripts already exist in the workflow skills (566 lines total): find_evidence.py, resolve_source.py, resolve_steps.py, parse_title.py, etc.

Specific problems with the current bash approach

1. jq crash on GitLab API responses (#118)

`jq` uses IEEE 754 doubles and crashes with `Invalid numeric literal` on large integers in GitLab project metadata. This broke fork detection entirely, causing MRs with empty diffs. Fixed in #118 by replacing jq with inline Python — but the pattern persists.

2. eval + inline Python = injection risk

Both `commit.sh` and `create_mr.sh` use `eval "$(python3 -c "...")"` to parse JSON and inject shell variables. If JSON values contain shell metacharacters, `eval` executes them. `shlex.quote()` mitigates this, but the entire pattern is unnecessary in pure Python.

3. readarray requires bash 4+ (macOS incompatible)

`commit.sh` uses `readarray` to parse the manifest. macOS ships bash 3.2. This silently fails or requires Homebrew bash.

4. JSON construction by string manipulation

`jira-ready-check.sh` builds JSON character-by-character with manual comma tracking. One edge case and the output is malformed. Python: `json.dumps(dict)`.

5. Untestable helper functions

`normalize_url()`, `write_step_result()`, `resolve_platform()` are embedded in bash scripts. No unit test can exercise them without spawning a full shell pipeline. Zero test coverage today.

6. 11 Python interpreter spawns per script run

`create_mr.sh` spawns `python3` 11 separate times for JSON operations. Each adds ~200ms of overhead. A single Python process handles all of this.

7. Inconsistent error handling

Some error paths write `step-result.json` before exiting, some do not. Bash's `set -euo pipefail` plus scattered `|| { echo ERROR; exit 1; }` is hard to reason about. Python's `try/except` with a single error-reporting function is explicit.

Benefits of Python rewrite

Benefit Example
Unified error handling Replace `set -euo pipefail` + scattered exit traps with `try/except` and a single error reporter
Testability Extract `normalize_url()`, `write_step_result()`, `parse_manifest()` as importable functions with unit tests
JSON handling Eliminate 11 `python3 -c` spawns per script — native `json.load()`/`json.dumps()`
Type safety `pushed` is a Python bool in JSON but compared as bash string `"true"` — fragile
Argument parsing Replace 30+ lines of `while/case/shift` boilerplate with `argparse` (5-10 lines)
macOS compatibility No bash version dependency (`readarray`, associative arrays)
Consistent logging `logging` module instead of ad-hoc `echo` to stdout/stderr

Proposed approach

Phase 1: target scripts (create_mr, commit, prepare_branch)

Rewrite as Python modules with a shared `workflow_utils.py`:

scripts/
  workflow_utils.py      # write_step_result, normalize_url, resolve_platform, parse_manifest
  create_mr.py
  commit.py
  prepare_branch.py
  tests/
    test_workflow_utils.py
    test_create_mr.py
  • Git and CLI commands via `subprocess.run(check=True, capture_output=True)`
  • Each script is `main`-invocable so SKILL.md files only change `.sh` → `.py`
  • Write unit tests for extracted functions (impossible today)
  • Estimated effort: 2-3 days

Phase 2: remaining scripts (create-jira-ticket, jira-ready-check, build_writing_args)

  • `create-jira-ticket.sh` already delegates to 3 Python helpers — the bash wrapper adds little value
  • `build_writing_args.sh` is pure arg parsing + JSON emission — straightforward rewrite
  • Estimated effort: 1-2 days

Risks

  • Git subprocess behavior: Git commands still run via `subprocess`. Risk is low — `subprocess.run(check=True)` is more explicit than `set -e`.
  • glab/gh CLI dependency: No Python SDK for `glab`, so these remain subprocess calls. `gh` could optionally use `PyGithub` but that adds a dependency.
  • Regression risk: Mitigated by writing tests first — which is itself impossible with the current bash scripts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions