test-flow - Testing workflow in Emacs Software is a conversation between what we intended and what actually runs. Tests translate between these two, but the loop “edit → run → read → fix → repeat” too often leaks our attention in tiny drips.
Test-Flow exists to keep the loop flowing. It watches your project, runs tests when it makes sense (not when you sneeze), parses whatever your runner emits, and shows a crisp, actionable panel near your code.
Right-side results panel grouped by suite; foldable sections with status icons.
Two runner modes:
external-command (default): robust, concurrent, parses stdout (JSON or ERT batch).
in-emacs-ert: synchronous, precise file/line/tags, instant goto-definition.
Parser modes: auto (JSON → batch fallback), JSON, or ERT batch.
Watcher: after-save or file-notify; debounce; auto-disable on idle sessions.
Concurrency: cap active runs; queue the rest; live counters in the Status block.
Header-line controls: run, run failed, watch toggle, detect command, copy failures, clear, goto, sessions, dashboard, logging toggle.
Copy failures: plain/org/markdown; includes full backtraces and optional stdout/stderr tails.
Multi-session aware: one panel per project; session list and dashboard across projects.
Unicode by default; optional all-the-icons for richer glyphs (with graceful degradation).
Defensive by design: parsers are tolerant, errors logged optionally, UI stays up.
Be invisible when all is well; be obvious and helpful when it isn’t.
Scale from tiny packages to larger suites by mixing in-Emacs speed with external-process robustness.
Offer humane ergonomics: a stable right-side panel, a readable header-line, clickable controls, and copy-pasteable failure reports.
Emacs 27 or newer.
Optional: all-the-icons for fancy toolbar/header icons.
(use-package test-flow
:init
(unless (package-installed-p 'test-flow)
(package-vc-install
'(test-flow
:vc-backend Git
:url "https://github.com/11111000000/test-flow"
:branch "main"))))
(use-package test-flow
:load-path "~/Code/test-flow/lisp"
:commands (test-flow-mode test-flow-open-panel)
:custom
(test-flow-run-on-open t))
Place test-flow.el, test-flow-headerline.el, test-flow-view-controls.el, test-flow-controls-icons.el on your load-path.
(require 'test-flow)
For all-the-icons: install the fonts and the all-the-icons package. If not present, test-flow uses Unicode/text gracefully.
In your project:
M-x test-flow-mode
M-x test-flow-open-panel
M-x test-flow-detect-runner (auto-finds tests/run-tests.el, nix flake, or Cask)
M-x test-flow-run
Panel keybindings (inside the results buffer):
r — Run tests
f — Run only failed (if external argv + failed-args are configured, else runs all)
w — Toggle watcher (after-save/file-notify per session config)
d — Detect runner
c — Copy last failures
x — Clear panel
o — Goto test definition
RET — Open details for test at point
TAB — Fold/unfold group (suite); S-TAB — fold/unfold all
n/p (or j/k) — Next/previous item
Filters: P/F/E/S/A — status filters; / — name regexp; T — tags; C — clear filters
Modular architecture and extension points
Entry points
test-flow.el — legacy monolith entry (stable public commands remain)
test-flow-modular.el — modular aggregator; (require ‘test-flow-modular) also provides ‘test-flow
Core modules (hard deps)
test-flow-core.el
Sessions: registry, cl-defstruct, safe accessors, per-session config (conf get/set)
Project root resolution, lightweight logging, utilities (suite-of, normalize-command, nix currentSystem)
test-flow-parse.el
Parsers: ERT batch и JSON; диспетчер test-flow-parse-output (‘auto|’json|’ert-batch)
Независим от UI/runner; устойчив к шумному выводу
test-flow-runner.el
Реализации очереди/процессов/сентинела, in-Emacs path; публичные команды
Хуки: test-flow-before-run-hook, test-flow-after-run-hook
test-flow-panel.el
Публичные команды панели (пока обёртки), фильтры, навигация
test-flow-render.el
Вынесен рендер: иконки/фейсы/счётчики, группировка/вставки, пайплайн рендера (context/insert/restore/render)
test-flow-watch.el
Реализации вотчера (after-save, file-notify, debounce, idle-GC); делегаты из монолита
test-flow-copy.el
Полноценная реализация copy-failures (monolith делегирует сюда)
Optional integrations/UI (soft deps)
test-flow-headerline.el — header-line renderer with caching and click handling
test-flow-view-controls.el — declarative controls and segment rendering
test-flow-controls-icons.el — graphic icons provider (all-the-icons)
test-flow-coverage.el — LCOV loader, overlays, panel block; auto-loads via test-flow-after-run-hook
test-flow-transient.el — global transient and keybindings
test-flow-i18n.el — localized labels/messages for controls and transient
Hooks and contracts
test-flow-before-run-hook (sess) — before a run starts
test-flow-after-run-hook (sess summary results) — after parsing and storage
Copy helpers accept user options: test-flow-copy-format, test-flow-copy-backtrace-limit, test-flow-copy-include-stdout/stderr
Migration notes
Monolith делегирует в модули (parse/watch/copy/runner/render); публичные имена команд сохранены
Панельный режим остаётся в монолите, но рендер целиком вынесен в test-flow-render (монолит делегирует)
Все опциональные модули тянутся мягко; без них доступен Unicode/text fallback
Sessions
One session per project (project.el root). Each holds: panel buffer, details buffer, watcher state, last results/summary, process, queue.
Runner modes
external-command: launches an external process; robust for long suites; concurrent across sessions; parses stdout/stderr.
in-emacs-ert: runs ERT in-process; precise metadata; synchronous (may block UI for long runs).
Parsers
auto tries JSON first; falls back to batch ERT. You can force JSON or batch explicitly.
Watchers
after-save: cheap and simple; runs after relevant buffers are saved.
file-notify: OS-level directory watches; configurable depth; include/exclude regexps.
Debounce and auto-disable-on-idle keep noise and CPU down.
Concurrency
Global cap on concurrent runs with FIFO queue. Status block shows “Proc: active N, queued M”.
Header-line controls
Clickable, icon/text-based, with mouse-1 and tooltips; reflects toggle states (watch/logging).
test-flow-mode — global minor mode; opens panel on enable.
test-flow-open-panel — open/focus session panel for current project.
test-flow-run — run tests according to per-session runner.
test-flow-run-failed — re-run failures/errors only (when available), else run all.
test-flow-detect-runner — detect external command (tests/run-tests.el, nix flake, Cask).
test-flow-toggle-watch — toggle watcher for current session.
test-flow-copy-failures — copy failures with backtraces (plain/org/markdown).
test-flow-clear — clear panel and last results.
test-flow-open-details-at-point — show details view for current test.
test-flow-goto-definition-at-point — jump to test function if loaded.
Navigation: test-flow-next-item, test-flow-previous-item, test-flow-toggle-group-at-point, test-flow-toggle-all-groups.
Filters: test-flow-panel-filter-pass/fail/error/skip/all, test-flow-panel-set-name-filter, test-flow-panel-set-tags-filter, test-flow-panel-filter-clear.
test-flow-list-sessions — list sessions with quick actions.
test-flow-switch-session — jump to another session’s panel.
test-flow-kill-session / test-flow-kill-all-sessions — clean up.
test-flow-dashboard — global overview (processes, sessions, summaries).
Debugging and maintenance
test-flow-toggle-logging — toggle lightweight logs.
test-flow-dump-concurrency — print concurrency state to Messages .
test-flow-restart — restart test-flow (cleans sessions/queues/timers, re-opens panel).
Configuration reference (customize these)
Variable Type Default Description test-flow-runner choice: external-command/in-emacs-ert external-command Backend to execute tests. test-flow-parser choice: auto/json/ert-batch auto Parsing strategy for test output. test-flow-external-command list argv or string (shell) nil External command to run tests. If string, executed via SHELL -lc. test-flow-external-failed-args-function function or nil nil (fn failed-names) → extra argv to run only failures (requires argv form).
Watcher and re-run ergonomics
Variable Type Default Description test-flow-watch-mode choice: after-save/file-notify/nil after-save How to watch the project for changes. test-flow-debounce-seconds number 0.7 Delay before running after a triggering change. test-flow-watch-include-regexp regexp or nil .el\’ Only paths matching this are eligible (nil = include all). test-flow-watch-exclude-regexp regexp or nil common dirs Exclude matching paths (.git, .direnv, node_modules, build, dist…). test-flow-file-notify-max-depth integer 3 Recursion depth for file-notify watchers. test-flow-session-idle-seconds integer 120 Auto-disable watch for idle sessions after this many seconds. test-flow-idle-gc-interval integer 30 Interval between idle GC checks.
Concurrency and resources
Variable Type Default Description test-flow-max-concurrent-runs integer 3 Global cap on parallel test processes (across sessions). test-flow-max-raw-output-bytes int or nil 1048576 Cap stored raw stdout/stderr per session (nil = unlimited).
Panel, UI, and header-line
Variable Type Default Description test-flow-panel-side choice: right/bottom/left/top right Where to display the side panel. test-flow-panel-width integer 42 Panel width in columns. test-flow-icons boolean t Show per-test status icons (Unicode or all-the-icons). test-flow-toolbar-style choice: auto/icons/text auto Prefer icons when available, force icons, or always text. test-flow-view-headerline-enable boolean t Show clickable controls in the panel’s header-line. test-flow-headerline-controls-order list of symbols/:gap (run run-failed :gap watch :gap copy clear :gap detect goto :gap sessions dashboard :gap logging) Order of header-line controls. test-flow-controls-registry alist (advanced) see code Declarative controls (labels/icons/commands).
Icon settings (if you use all-the-icons)
Variable Type Default Description test-flow-controls-use-graphic-icons boolean t Prefer all-the-icons when available. test-flow-controls-icon-height number 0.9 Uniform icon height in header-line. test-flow-controls-icon-raise number 0.11 Vertical raise via ‘display’ property (fine-tunes alignment). test-flow-controls-icon-map alist mapping Control-key → (provider . name) or per-state map. test-flow-controls-icon-face-map alist mapping Optional face overrides for non-toggle icons. test-flow-controls-toggle-on-face face/plist gray85 Face for toggle icons when ON. test-flow-controls-toggle-off-face face/plist gray60 Face for toggle icons when OFF.
Copy failures and reporting
Variable Type Default Description test-flow-copy-format choice: plain/org/markdown plain Format of copied failure report. test-flow-copy-backtrace-limit int or nil nil Truncate each details/backtrace to this many chars. test-flow-copy-include-stdout boolean nil Include raw stdout tail (capped by the same limit). test-flow-copy-include-stderr boolean nil Include captured stderr tail (capped by the same limit).
Variable Type Default Description test-flow-run-on-enable boolean nil If non-nil, run once when test-flow-mode is enabled. test-flow-auto-detect-on-open boolean t Try to auto-detect external command when opening the panel. test-flow-run-on-open boolean t First open triggers a run when feasible. test-flow-log-enabled boolean nil Print lightweight logs to Messages (toggled via command). test-flow-session-naming-function function default (fn root) → name for “*test-flow: NAME/” buffer.
(setq test-flow-external-command ‘(“emacs” “-Q” “–batch” “-l” “tests/run-tests.el”))
Per-project .dir-locals (session-level settings) ((lisp-mode
(test-flow-runner . external-command)
(test-flow-external-command . ("emacs" "-Q" "--batch" "-l" "tests/run-tests.el"))
(test-flow-parser . auto)
(test-flow-watch-mode . after-save)
(test-flow-debounce-seconds . 0.5)
(test-flow-file-notify-max-depth . 2)))
Run only failed externally (example adapter) (setq test-flow-external-failed-args-function
(lambda (names)
(when names
(list "--" "--tests" (mapconcat #'identity names ",")))))
JSON output (recommended schema)
summary: {total, passed, failed, error, skipped, duration_ms?, time?}
tests: array of {name, status, message?, details?, file?, line?, tags?}
{
"summary": {"total": 12, "passed": 10, "failed": 1, "error": 1, "duration_ms": 8342},
"tests": [
{"name":"ns/test-1","status":"pass"},
{"name":"ns/test-2","status":"fail","message":"expected X","details":"..."}
]
}
status is case-insensitive and accepts pass/ok, fail/failed, error, skip/skipped, xfail.
If duration_ms is missing, test-flow computes elapsed time when possible.
Using the panel effectively
Counters, duration, active/queued processes, project, runner, mode, watch state, parser.
Fold green-all-pass groups (auto-initialized); expand for failures/errors.
Click a test to see details; press o to jump to its definition (if loaded).
Header-line controls
Mouse-1 on icons: run, run failed, toggle watch, detect, copy, clear, goto, sessions, dashboard, logging.
Tooltips explain each control. If icons aren’t available, text labels are used.
Runner detection (external)
M-x test-flow-detect-runner tries:
tests/run-tests.el or test/run-tests.el → emacs -Q –batch -l <path>
flake.nix → nix run .#tests
Cask → cask exec ert-runner
If multiple entrypoints are found, you’ll be prompted to pick one.
Prefer in-emacs-ert to quickly jump to failures while iterating on a test file, switch to external for big suites.
Set a small debounce (0.3–0.7s) to keep the flow without running on every keystroke-save combo.
Use filters (P/F/E/S, / regexp, T tags) to focus on what matters right now.
Copy failures in org format for issue trackers that love org’s structured blocks. Your future self (and coworkers) will thank you.
Icons look plain
That’s okay! Unicode mode is intentional. Install all-the-icons for fancier looks. If your fonts and ligatures behave, Emacs will too (most of the time).
in-emacs-ert freezes Emacs
It’s synchronous by design. For long suites, pick external-command. Coffee is optional but recommended.
“Run failed” still runs everything
Ensure test-flow-external-command is a list (argv), not a shell string, and set test-flow-external-failed-args-function.
Nothing happens on save
Check test-flow-watch-mode, include/exclude regexps, and whether your file is under the project root (project.el).
JSON parser fails mysteriously
Keep batch fallback via ‘auto’. If your runner prints banners around JSON, test-flow tries to snip “{…}” out; when in doubt, emit a clean JSON blob.
Faces (customize for your theme)
Result faces
test-flow-face-pass, test-flow-face-fail, test-flow-face-error, test-flow-face-skip
Toolbar/header-line faces
test-flow-headerline
Icon faces: test-flow-controls-icon-on/off, or overrides via the icon face map.
Toolbar button faces (legacy in-buffer toolbar is superseded by header-line controls, but faces remain available)
Developer notes (optional)
Header-line controls are declared in test-flow-view-controls.el; icons live in test-flow-controls-icons.el; the renderer and cache are in test-flow-headerline.el.
External runs use make-process with stderr captured separately; sentinel selects stdout vs stderr for parsing, trims buffers according to test-flow-max-raw-output-bytes.
In-Emacs runs enrich results directly from ERT objects: file/line, tags, and backtrace pretty-printing.
Issues and PRs are welcome. Please include:
Emacs version, OS, and how you run tests (external vs in-Emacs).
A snippet of stdout/stderr or JSON (trimmed is fine) when parsing is the issue.
A screenshot of the panel if a UI quirk is suspected (optional but delightful).
You can run package tests in batch:
emacs -Q –batch -L lisp -l test-flow.el -l tests/test-flow-tests.el -f ert-run-tests-batch-and-exit
Or via your project’s tests/run-tests.el.
MIT. Share, remix, and don’t blame us if your tests discover new truths about your code.
In-Emacs reporter
Capture precise file/line/tags/duration without text parsing; unify with in-emacs-ert path.
Dashboard++
Filterable, sortable multi-session view; quick actions; persistent layout.
Parsers
TAP and pluggable custom parsers; richer JSON schema (attachments, artifacts).
UX
Status/search in panel; better folding persistence; richer filters (by tag/status/duration).
Adaptive ergonomics
Dynamic panel width (golden ratio option); auto-switch runner based on suite size/duration.
Packaging
MELPA recipe; more examples for JSON emitters; improved Nix story and templates.
Documentation
More “recipes” for common project setups; troubleshooting playbook with patterns.
