From d1526418722ad4fc800e1d542767780482ac9039 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:02:17 +0000 Subject: [PATCH 01/12] docs: map existing codebase --- .planning/codebase/ARCHITECTURE.md | 80 +++++++++++++++++++++ .planning/codebase/CONCERNS.md | 76 ++++++++++++++++++++ .planning/codebase/CONVENTIONS.md | 75 ++++++++++++++++++++ .planning/codebase/INTEGRATIONS.md | 51 ++++++++++++++ .planning/codebase/STACK.md | 60 ++++++++++++++++ .planning/codebase/STRUCTURE.md | 109 +++++++++++++++++++++++++++++ .planning/codebase/TESTING.md | 73 +++++++++++++++++++ 7 files changed, 524 insertions(+) create mode 100644 .planning/codebase/ARCHITECTURE.md create mode 100644 .planning/codebase/CONCERNS.md create mode 100644 .planning/codebase/CONVENTIONS.md create mode 100644 .planning/codebase/INTEGRATIONS.md create mode 100644 .planning/codebase/STACK.md create mode 100644 .planning/codebase/STRUCTURE.md create mode 100644 .planning/codebase/TESTING.md diff --git a/.planning/codebase/ARCHITECTURE.md b/.planning/codebase/ARCHITECTURE.md new file mode 100644 index 0000000..84f138d --- /dev/null +++ b/.planning/codebase/ARCHITECTURE.md @@ -0,0 +1,80 @@ +# Architecture + +## Pattern + +**Layered controller architecture** built on FastCS framework. + +The system follows a hierarchical controller pattern where a top-level `CATioController` discovers EtherCAT hardware via ADS, creates sub-controllers for each device/terminal, and exposes their attributes as EPICS PVs through FastCS. + +## Layers + +``` +┌─────────────────────────────────────────────┐ +│ EPICS / NiceGUI (Presentation) │ +│ FastCS backends: softioc, nicegui │ +├─────────────────────────────────────────────┤ +│ FastCS Framework (Attribute Layer) │ +│ AttrR, AttrRW, AttrW → PV mapping │ +├─────────────────────────────────────────────┤ +│ CATio Controllers (Domain Layer) │ +│ CATioController → Device/Terminal subs │ +│ Hardware-specific controllers (EL3xxx etc) │ +├─────────────────────────────────────────────┤ +│ CATio Dynamic Layer │ +│ Dynamic CoE, Symbol, Type controllers │ +│ YAML-driven terminal configuration │ +├─────────────────────────────────────────────┤ +│ ADS Client (Transport Layer) │ +│ Async TCP/UDP, AMS protocol, routing │ +├─────────────────────────────────────────────┤ +│ TwinCAT PLC / EtherCAT Hardware │ +└─────────────────────────────────────────────┘ +``` + +## Key Abstractions + +### Controllers (`src/fastcs_catio/catio_controller.py`) +- `CATioController` — Base controller with ADS connection management, attribute creation, notification handling +- `CATioDeviceController` — Sub-controller for EtherCAT master devices +- `CATioTerminalController` — Sub-controller for EtherCAT slave terminals + +### Hardware Controllers (`src/fastcs_catio/catio_hardware.py`) +- Terminal-specific controllers: `EL1xxx`, `EL2xxx`, `EL3xxx`, `EL4xxx`, `EL3702` (oversampling) +- Each defines channel counts, I/O functions, attribute mappings + +### Dynamic Controllers (`src/fastcs_catio/catio_dynamic_controller.py`, `catio_dynamic_coe.py`, `catio_dynamic_symbol.py`) +- Runtime-generated controllers from YAML terminal type definitions +- CoE (CANopen over EtherCAT) object discovery and attribute creation +- Symbol-based dynamic attribute creation + +### ADS Client (`src/fastcs_catio/client.py`) +- `CatioClient` — Full async ADS client with TCP/UDP transport +- Connection management, route creation/deletion +- Symbol table reading, notification subscriptions +- Device/slave introspection + +### Attribute I/O (`src/fastcs_catio/catio_attribute_io.py`) +- `CATioControllerAttributeIO` — Bridge between FastCS attributes and ADS read/write +- Handles polling periods, notification-based updates, value caching + +## Data Flow + +1. **Startup:** CLI creates `CATioController` → connects via ADS → discovers devices/terminals +2. **Introspection:** For each device/terminal, creates sub-controllers → reads attributes from ADS +3. **Dynamic:** Loads YAML terminal configs → creates dynamic CoE/symbol controllers +4. **Runtime:** FastCS runs scan loop → polls ADS for attribute updates → pushes to EPICS PVs +5. **Notifications:** ADS notifications stream → update controller attributes → propagate to EPICS + +## Entry Points + +- `src/fastcs_catio/__main__.py` — Typer CLI app with `run` command, connects to ADS and starts FastCS +- `src/catio_terminals/__main__.py` — NiceGUI web app for terminal configuration browsing +- `tests/ads_sim/__main__.py` — ADS simulation server for testing + +## Second Package: catio-terminals + +Separate package in `src/catio_terminals/` providing a web-based tool for: +- Browsing Beckhoff ESI XML terminal catalogs +- Viewing PDO mappings, CoE objects, symbol definitions +- Managing terminal type YAML configuration files +- Tree-view navigation of terminal hierarchies diff --git a/.planning/codebase/CONCERNS.md b/.planning/codebase/CONCERNS.md new file mode 100644 index 0000000..212fe09 --- /dev/null +++ b/.planning/codebase/CONCERNS.md @@ -0,0 +1,76 @@ +# Concerns + +## Tech Debt + +### High Priority + +1. **Single EtherCAT device hardcoded** — `src/fastcs_catio/client.py` has an assertion that blocks multi-master device support. Only one device can be used at a time. + +2. **All dynamic symbol types map to `Int()`** — `src/fastcs_catio/terminal_config.py:151` has a TODO: all symbols map to `Int()` regardless of actual type. Should map using `symbol.type_name`. + +3. **Route deletion commented out** — `src/fastcs_catio/catio_controller.py:456` route cleanup on disconnect is disabled. + +4. **Test suites entirely skipped** — `test_catio_system.py` and `test_catio_performance.py` are both skipped at module level with `reason="TODO these are all failing"`. + +### Medium Priority + +5. **Global mutable poll period variables** — `src/fastcs_catio/catio_controller.py:390-393` — shared across all controller instances. + +6. **Class-level TCP connection** — `src/fastcs_catio/catio_controller.py:63` — single connection shared across all instances. + +7. **Notification handle uniqueness** — `src/fastcs_catio/client.py:2549` — not enforced across devices. + +8. **`EL3104Controller.read_configuration()` stub** — `src/fastcs_catio/catio_hardware.py:681` — uses `print()` instead of logging, says "NOT IMPLEMENTED YET". + +9. **Oversampling factor hardcoded** — `src/fastcs_catio/catio_hardware.py:815-819` — `OVERSAMPLING_FACTOR` constant instead of reading from ADS. Has a TODO comment. + +10. **CoE attribute name collision check incomplete** — `src/fastcs_catio/catio_dynamic_controller.py:123`. + +## Known Bugs + +1. **`_recv_forever` silent exit** — The receive loop exits silently on `AssertionError`, killing the receive task without surfacing errors to the user. + +2. **`get_io_from_map()` raises `KeyError`** — On second call, raises `KeyError` instead of returning cached value. + +3. **`UDPMessage.invoke_id` is non-thread-safe** — Class-level counter incremented without locking. + +## Security + +1. **Default password `"1"` hardcoded** — `RemoteRoute` uses a default password of `"1"` for ADS route authentication. + +2. **UDP response origin not verified** — No cryptographic verification of ADS UDP responses. + +3. **Password transmitted in plaintext** — ADS route passwords sent as plaintext UDP bytes. + +> **Context:** These are inherent limitations of the ADS protocol rather than implementation oversights. ADS is designed for closed industrial networks, not public-facing systems. + +## Performance + +1. **`average()` non-vectorised loop** — GitHub issue #22 open. Manual loop instead of numpy vectorized operation. + +2. **Sequential per-slave ADS round-trips on startup** — Device/terminal introspection happens sequentially, could be parallelized. + +3. **Unconditional notification array averaging** — Runs every scan cycle regardless of whether new data arrived. + +## Fragile Areas + +1. **Bare `except Exception` in `_recv_ams_message`** — Catches and logs all exceptions, potentially hiding protocol errors. + +2. **Bare `except Exception` in `notifications()` scan** — Same pattern in notification processing. + +3. **`symbol_lookup()` manual LUT** — Has multiple `TO DO: REVIEW` branch markers suggesting incomplete logic. + +4. **Dynamic controller cache** — Not invalidated on error, may serve stale data. + +5. **`CATioControllerSymbolAttributeIO.update()`** — Is a `pass` stub (no-op). + +## Scaling Limits + +- **Single master device** enforced at two assertion points in `client.py` +- **Unbounded notification queue** — no backpressure mechanism + +## Missing Features + +- No TCP reconnection logic (connection drops require restart) +- No route cleanup CLI command +- No graceful degradation on partial device failures diff --git a/.planning/codebase/CONVENTIONS.md b/.planning/codebase/CONVENTIONS.md new file mode 100644 index 0000000..0b5a3a5 --- /dev/null +++ b/.planning/codebase/CONVENTIONS.md @@ -0,0 +1,75 @@ +# Conventions + +## Code Style + +- **Formatter/Linter:** ruff (line-length 88) +- **Lint rules:** B (bugbear), C4 (comprehensions), E (pycodestyle errors), F (pyflakes), N (naming), W (warnings), I (isort), UP (pyupgrade), SLF (private member access) +- **Type checking:** pyright in standard mode +- **Python:** 3.11+ features used (match statements, `str | None` unions) + +## Naming + +| Element | Convention | Example | +|---------|-----------|---------| +| Files | snake_case, `catio_` prefix for core | `catio_controller.py` | +| Classes | PascalCase, `CATio` prefix | `CATioController`, `CATioDeviceController` | +| Hardware classes | Part number + Controller | `EL3104Controller` | +| Functions/methods | snake_case | `get_io_attributes()`, `read_configuration()` | +| Constants | UPPER_SNAKE_CASE | `OVERSAMPLING_FACTOR`, `TWINCAT_STRING_ENCODING` | +| Private | Leading underscore | `_recv_forever()`, `_ecdevices` | +| Test files | `test_` prefix | `test_catio_units.py` | + +## Patterns + +### Controller Hierarchy +Controllers follow FastCS `SubController` pattern: +```python +class CATioDeviceController(CATioController): + async def get_io_attributes(self) -> None: + await self.get_device_generic_attributes() +``` + +### Attribute Registration +Attributes are added via `self.add_attribute()` with FastCS types: +```python +self.add_attribute( + "Name", + AttrR( + datatype=String(), + io_ref=CATioControllerAttributeIORef("name", update_period=ONCE), + group=self.attr_group_name, + initial_value=self.io.name, + description="I/O device name", + ), +) +``` + +### Async Throughout +All I/O operations are async. Uses `asyncio` event loops, `asyncio.Task`, `asyncio.Event` for coordination. + +### Dataclass-style Config +Hardware controllers use class-level attributes for configuration: +```python +class EL3104Controller(CATioTerminalController): + io_function: str = "4-channel analog input, +/-10V, 16-bit, differential" + num_channels: int = 4 +``` + +## Error Handling + +- `assert isinstance(...)` used for type narrowing in controller methods +- `logger.exception()` for caught errors in async loops +- `logger.warning()` for degraded but functional states +- Bare `except Exception` in some notification/receive loops (flagged as concern) + +## Logging + +- Custom `VERBOSE` level (below DEBUG) defined in `src/fastcs_catio/logging.py` +- `VerboseLogger` subclass with `logger.verbose()` method +- Standard `getLogger(__name__)` pattern throughout + +## Documentation + +- Sphinx with MyST parser, pydata theme +- Docstrings use reStructuredText (`:param:`, `:returns:`) +- Doctest enabled in pytest config diff --git a/.planning/codebase/INTEGRATIONS.md b/.planning/codebase/INTEGRATIONS.md new file mode 100644 index 0000000..359f616 --- /dev/null +++ b/.planning/codebase/INTEGRATIONS.md @@ -0,0 +1,51 @@ +# Integrations + +## External Protocols + +### ADS (Automation Device Specification) — Primary +- **Protocol:** TwinCAT ADS over TCP (port 48898) and UDP (port 48899) +- **Implementation:** Custom async client in `src/fastcs_catio/client.py` (~2500 lines) +- **Purpose:** Communicate with Beckhoff TwinCAT PLCs and EtherCAT I/O devices +- **Key operations:** Read/write symbols, device introspection, notification subscriptions, route management +- **Message layer:** `src/fastcs_catio/messages.py` — AMS/TCP and AMS headers, request/response framing + +### EPICS — Control System Backend +- **Protocol:** Channel Access (CA) / PVAccess (PVA) via softioc +- **Integration:** FastCS handles EPICS backend automatically via `fastcs[epics]` +- **Purpose:** Expose EtherCAT I/O as EPICS Process Variables + +### HTTP — Beckhoff XML Catalog +- **Client:** httpx in `src/catio_terminals/beckhoff.py` +- **Purpose:** Download EtherCAT Slave Information (ESI) XML files from Beckhoff's online catalog +- **Endpoint:** Beckhoff's public ESI XML repository + +## Internal Services + +### ADS Simulation Server +- **Location:** `tests/ads_sim/` +- **Purpose:** Test harness simulating a TwinCAT ADS server with configurable EtherCAT chains +- **Entry point:** `tests/ads_sim/__main__.py` — standalone async TCP/UDP server +- **Protocol:** Full ADS/AMS protocol simulation + +### NiceGUI Web Application +- **Location:** `src/catio_terminals/ui_app.py` +- **Purpose:** Web-based terminal configuration and browsing tool +- **Components:** `src/catio_terminals/ui_components/` — tree views, detail panes, symbol details +- **Dialogs:** `src/catio_terminals/ui_dialogs/` — confirmation, file, database, terminal dialogs + +## Data Stores + +### YAML Configuration Files +- Terminal type definitions: `src/catio_terminals/terminals/terminal_types.yaml` +- Runtime symbols: `src/catio_terminals/config/runtime_symbols.yaml` +- Controller config: `src/fastcs_catio/catio_controller.yaml` + +### ESI XML Files +- Parsed by `src/catio_terminals/xml/` package +- Contains: PDO definitions, CoE objects, device descriptions +- Cached locally after download from Beckhoff catalog + +## Authentication + +- ADS routes use password-based authentication (default password: `"1"` in `RemoteRoute`) +- No other auth mechanisms (no OAuth, no API keys) diff --git a/.planning/codebase/STACK.md b/.planning/codebase/STACK.md new file mode 100644 index 0000000..493f905 --- /dev/null +++ b/.planning/codebase/STACK.md @@ -0,0 +1,60 @@ +# Stack + +## Language & Runtime + +| Property | Value | +|----------|-------| +| Language | Python | +| Version | >=3.11 (3.11, 3.12, 3.13) | +| Build System | setuptools + setuptools_scm | +| Package Manager | uv (lockfile: `uv.lock`) | + +## Core Dependencies + +| Package | Version | Purpose | +|---------|---------|---------| +| fastcs | 0.12.0a1 (with `[epics]` extra) | Control system framework — provides `Controller`, `SubController`, `AttrR`, `AttrRW`, `AttrW` and backend runners | +| softioc | >=4.7.0 | EPICS IOC server (Channel Access / PVAccess) | +| numpy | (latest) | Numeric data handling, array operations for oversampling | +| pvi | (latest) | Process Variable Interface generation | +| nicegui | >=3.6.1 | Web UI framework for `catio-terminals` tool | +| typer | (latest) | CLI framework for both entry points | + +## Optional Dependencies (`[terminals]`) + +| Package | Version | Purpose | +|---------|---------|---------| +| nicegui | >=2.0.0 | Web UI (also core dep) | +| pydantic | >=2.0.0 | Data models for terminal configuration | +| pyyaml | >=6.0 | YAML parsing | +| ruamel.yaml | >=0.18.0 | YAML read/write with comment preservation | +| httpx | >=0.27.0 | HTTP client for Beckhoff XML catalog downloads | + +## Dev Dependencies + +| Package | Purpose | +|---------|---------| +| pytest + pytest-asyncio + pytest-cov | Testing | +| ruff | Linting & formatting | +| pyright | Type checking | +| pre-commit | Git hooks | +| sphinx + extensions | Documentation | +| tox-uv | Test automation | + +## Entry Points + +| Command | Module | Purpose | +|---------|--------|---------| +| `fastcs-catio` | `fastcs_catio.__main__:app` | Main control system application (Typer CLI) | +| `catio-terminals` | `catio_terminals.__main__:main` | Terminal configuration web UI | + +## Configuration + +- `pyproject.toml` — project metadata, dependencies, tool config +- `src/fastcs_catio/catio_controller.yaml` — controller configuration +- `src/catio_terminals/config/runtime_symbols.yaml` — runtime symbol definitions +- `src/catio_terminals/terminals/terminal_types.yaml` — terminal type definitions +- `renovate.json` — dependency update automation +- `ruff` — line-length 88, rules: B, C4, E, F, N, W, I, UP, SLF +- `pyright` — standard mode, `reportMissingImports = false` +- `pytest` — asyncio auto mode, doctest enabled diff --git a/.planning/codebase/STRUCTURE.md b/.planning/codebase/STRUCTURE.md new file mode 100644 index 0000000..8f98bbe --- /dev/null +++ b/.planning/codebase/STRUCTURE.md @@ -0,0 +1,109 @@ +# Structure + +## Directory Layout + +``` +fastcs-catio/ +├── src/ +│ ├── fastcs_catio/ # Main package — EtherCAT control system +│ │ ├── __init__.py +│ │ ├── __main__.py # Typer CLI entry point +│ │ ├── _constants.py # Constants (encoding, defaults) +│ │ ├── _types.py # Type aliases +│ │ ├── _version.py # Auto-generated version (setuptools_scm) +│ │ ├── catio_attribute_io.py # FastCS attribute ↔ ADS I/O bridge +│ │ ├── catio_connection.py # Connection management +│ │ ├── catio_controller.py # Core controller hierarchy (~1200 lines) +│ │ ├── catio_controller.yaml # Controller configuration +│ │ ├── catio_dynamic_coe.py # Dynamic CoE object controllers +│ │ ├── catio_dynamic_controller.py # Dynamic terminal controllers +│ │ ├── catio_dynamic_symbol.py # Dynamic symbol controllers +│ │ ├── catio_dynamic_types.py # Dynamic type definitions +│ │ ├── catio_hardware.py # Hardware-specific controllers (EL series) +│ │ ├── client.py # ADS async client (~2500 lines, largest file) +│ │ ├── devices.py # Device/slave data models +│ │ ├── logging.py # Custom VERBOSE logging level +│ │ ├── messages.py # AMS/TCP protocol messages +│ │ ├── symbols.py # ADS symbol handling +│ │ ├── terminal_config.py # YAML terminal type configuration +│ │ └── utils.py # Network/data utilities +│ │ +│ └── catio_terminals/ # Second package — terminal config web UI +│ ├── __init__.py +│ ├── __main__.py # NiceGUI web app entry point +│ ├── ads_types.py # ADS type definitions +│ ├── beckhoff.py # Beckhoff ESI XML catalog client +│ ├── models.py # Pydantic data models +│ ├── service_config.py # Configuration service +│ ├── service_file.py # File management service +│ ├── service_terminal.py # Terminal service +│ ├── ui_app.py # Main NiceGUI application +│ ├── utils.py # Utility functions +│ ├── config/ +│ │ └── runtime_symbols.yaml +│ ├── terminals/ +│ │ └── terminal_types.yaml +│ ├── ui_components/ # NiceGUI components +│ │ ├── details_pane.py +│ │ ├── symbol_details.py +│ │ ├── terminal_details.py +│ │ ├── tree_data_builder.py +│ │ ├── tree_view.py +│ │ └── utils.py +│ ├── ui_dialogs/ # NiceGUI dialogs +│ │ ├── confirmation_dialogs.py +│ │ ├── database_dialogs.py +│ │ ├── delete_dialogs.py +│ │ ├── file_dialogs.py +│ │ └── terminal_dialogs.py +│ └── xml/ # ESI XML parsing +│ ├── cache.py +│ ├── catalog.py +│ ├── coe.py +│ ├── constants.py +│ ├── parser.py +│ ├── pdo_groups.py +│ └── pdo.py +│ +├── tests/ +│ ├── conftest.py # Shared fixtures +│ ├── mock_server.py # Mock ADS server for unit tests +│ ├── ads_sim/ # Full ADS simulation server +│ │ ├── __main__.py # Standalone entry point +│ │ ├── ethercat_chain.py # Simulated EtherCAT device chains +│ │ └── server.py # Async TCP/UDP ADS server +│ ├── test_async_client.py # ADS client tests +│ ├── test_catio_system.py # System tests (currently skipped) +│ ├── test_catio_performance.py # Performance tests (currently skipped) +│ ├── test_catio_dynamic.py # Dynamic controller tests +│ ├── test_catio_terminals.py # Terminal package tests +│ ├── test_catio_units.py # Unit tests +│ ├── test_cli.py # CLI tests +│ └── ... (13 more test files) +│ +├── docs/ # Sphinx documentation +├── pyproject.toml # Project configuration +├── uv.lock # Dependency lockfile +└── renovate.json # Dependency automation +``` + +## Key Locations + +| What | Where | +|------|-------| +| Main controller logic | `src/fastcs_catio/catio_controller.py` | +| ADS protocol client | `src/fastcs_catio/client.py` | +| Hardware controllers | `src/fastcs_catio/catio_hardware.py` | +| Dynamic controllers | `src/fastcs_catio/catio_dynamic_*.py` | +| Terminal web UI | `src/catio_terminals/ui_app.py` | +| Test fixtures | `tests/conftest.py` | +| ADS simulator | `tests/ads_sim/` | + +## Naming Conventions + +- **Files:** snake_case, prefixed with `catio_` for core module files +- **Classes:** PascalCase, prefixed with `CATio` for controllers +- **Hardware classes:** Named after Beckhoff part numbers (e.g., `EL3104Controller`) +- **Dynamic prefixes:** `catio_dynamic_` for runtime-generated controller modules +- **UI organization:** `ui_components/` and `ui_dialogs/` sub-packages +- **Test files:** `test_` prefix, descriptive names diff --git a/.planning/codebase/TESTING.md b/.planning/codebase/TESTING.md new file mode 100644 index 0000000..3acf70b --- /dev/null +++ b/.planning/codebase/TESTING.md @@ -0,0 +1,73 @@ +# Testing + +## Framework + +| Tool | Version | Purpose | +|------|---------|---------| +| pytest | (latest) | Test runner | +| pytest-asyncio | >=1.3.0 | Async test support (auto mode) | +| pytest-cov | (latest) | Coverage reporting | +| tox-uv | (latest) | Test automation with uv | + +## Configuration + +```toml +# pyproject.toml +asyncio_mode = "auto" +testpaths = "docs src tests" +addopts = "--tb=native -vv --doctest-modules --doctest-glob='*.rst'" +``` + +## Test Structure + +``` +tests/ +├── conftest.py # Shared fixtures, pytest configuration +├── mock_server.py # Mock ADS server for unit-level tests +├── ads_sim/ # Full ADS simulation server +│ ├── __main__.py # Standalone entry point +│ ├── ethercat_chain.py # Configurable simulated EtherCAT chains +│ └── server.py # Async TCP/UDP ADS protocol server +├── test_async_client.py # ADS client async tests +├── test_beckhoff_client.py # Beckhoff XML catalog tests +├── test_catio_dynamic.py # Dynamic controller tests +├── test_catio_performance.py # Performance tests (SKIPPED) +├── test_catio_system.py # System integration tests (SKIPPED) +├── test_catio_terminals.py # Terminal package tests +├── test_catio_units.py # Core unit tests +├── test_cli.py # CLI command tests +├── test_coe_readwrite.py # CoE read/write tests +├── test_mock_server_example.py # Mock server usage example +├── test_new_terminal_coe_selection.py # Terminal CoE selection tests +├── test_pdo_groups.py # PDO group tests +├── test_symbol_notifications.py # Symbol notification tests +├── test_symbol_readwrite.py # Symbol read/write tests +├── test_system.py # System-level tests +├── test_variable_polling.py # Variable polling tests +└── test_xml_parser_pdo.py # XML PDO parser tests +``` + +## Test Approaches + +### ADS Simulation Server (`tests/ads_sim/`) +Full async ADS protocol simulator used for integration tests. Supports configurable EtherCAT device chains with simulated I/O. Runs as standalone process or within test fixtures. + +### Mock Server (`tests/mock_server.py`) +Lighter-weight mock for unit tests that don't need full protocol simulation. + +### Async Testing +All tests use `pytest-asyncio` in auto mode — async test functions and fixtures are automatically detected. + +## Coverage + +- Coverage via `pytest-cov`, reporting to `/tmp/fastcs_catio.coverage` +- Source mapping from installed packages back to `src/` +- XML report: `cov.xml` + +## Known Gaps + +- `test_catio_system.py` — **entirely skipped** (`pytest.skip(allow_module_level=True, reason="TODO these are all failing")`) +- `test_catio_performance.py` — **entirely skipped** (same reason) +- Error recovery paths in `_recv_forever` untested +- Symbol refresh flow untested +- Multi-stream notification paths untested From b798e012b99e2748ac1986bcb7db44c50a327618 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:05:31 +0000 Subject: [PATCH 02/12] chore: add GSD project config and gitignore .claude/ Co-Authored-By: Claude Opus 4.6 --- .gitignore | 3 +++ .planning/config.json | 16 ++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 .planning/config.json diff --git a/.gitignore b/.gitignore index f4c7ed5..16d5496 100644 --- a/.gitignore +++ b/.gitignore @@ -75,3 +75,6 @@ screens .test.yaml .nicegui + +# Claude Code / GSD tooling +.claude/ diff --git a/.planning/config.json b/.planning/config.json new file mode 100644 index 0000000..07b6fe2 --- /dev/null +++ b/.planning/config.json @@ -0,0 +1,16 @@ +{ + "model_profile": "quality", + "commit_docs": true, + "search_gitignored": false, + "branching_strategy": "none", + "phase_branch_template": "gsd/phase-{phase}-{slug}", + "milestone_branch_template": "gsd/{milestone}-{slug}", + "workflow": { + "research": true, + "plan_check": true, + "verifier": true, + "nyquist_validation": false + }, + "parallelization": true, + "brave_search": false +} \ No newline at end of file From 44e36fa017eaafefc7949667a5c2904069d09604 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:24:20 +0000 Subject: [PATCH 03/12] docs: initialize project --- .planning/PROJECT.md | 62 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 .planning/PROJECT.md diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md new file mode 100644 index 0000000..276e5d0 --- /dev/null +++ b/.planning/PROJECT.md @@ -0,0 +1,62 @@ +# AI Agents in CATio — Executive Summary Document + +## What This Is + +A documentation page for the CATio Sphinx docs (in `docs/explanations/`) that provides an executive summary of how AI agents (Claude Opus) were used to deliver a major architectural overhaul of the CATio project in approximately two weeks. Targeted at management and stakeholders, making the case for AI-assisted development. + +## Core Value + +Demonstrate through concrete evidence that AI-assisted development enabled a single developer to deliver architectural changes of a scale and quality that would typically require a larger team or significantly more time. + +## Requirements + +### Validated + +(None yet — ship to validate) + +### Active + +- [ ] High-level overview of the architectural changes (YAML-driven dynamic controllers replacing hard-coded Python classes, catio-terminals GUI) +- [ ] Quantified scale of changes (113 commits, ~23k lines new source, 16 to 56 source files, 29 terminal types in YAML) +- [ ] Analysis of where Claude was used to achieve rapid development (code generation, refactoring, module decomposition) +- [ ] Before/after comparison showing architecture and maintainability improvements +- [ ] Acknowledgement that this work predates agent skills and structured workflows +- [ ] Reference to builder2ibek as an example of more mature AI agent usage with proper agent features +- [ ] Advocacy tone — making the case for AI-assisted development based on this experience +- [ ] Follows existing Sphinx/MyST markdown format used by other explanation pages + +### Out of Scope + +- Test infrastructure and simulator changes — focus on architecture only +- Documentation commits and cleanup — focus on the code architecture +- Step-by-step tutorial on using AI agents — this is an executive summary +- Detailed code examples — keep it high-level for stakeholder audience + +## Context + +- The CATio project controls EtherCAT I/O devices via Beckhoff TwinCAT +- Before commit df5c3c4, terminal handling required explicit Python classes (~1200 lines in catio_hardware.py per terminal family) +- After the changes: terminal types are defined in YAML (14,276 lines covering 29 terminal types), generated from vendor Beckhoff XML, and editable via a NiceGUI web application +- Dynamic controller generation creates FastCS controller classes at runtime from YAML definitions +- The entire body of work was done by Giles Knap over ~2 weeks using Claude Opus +- 100 commits by Giles, 11 by copilot-swe-agent for doc updates, 2 by Gregory Gay +- The work predates use of GSD workflows, agent skills, and other structured AI development features +- builder2ibek (sibling project) is a migration tool built with more mature AI agent features and serves as a better example of the full agent workflow + +## Constraints + +- **Format**: Must be a Sphinx/MyST markdown page in `docs/explanations/` +- **Audience**: Management and stakeholders, not developers +- **Tone**: Advocacy — make the case based on evidence +- **Length**: Executive summary — concise, not exhaustive + +## Key Decisions + +| Decision | Rationale | Outcome | +|----------|-----------|---------| +| Architecture focus only | Stakeholders care about what changed and why, not test/doc commits | — Pending | +| Advocacy tone | User wants to make the case for AI-assisted development | — Pending | +| Reference builder2ibek | Shows trajectory of improvement in AI agent usage | — Pending | + +--- +*Last updated: 2026-03-02 after initialization* From 89e47493fdba382908029d3b5bbc473062934e25 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:26:04 +0000 Subject: [PATCH 04/12] chore: add project config --- .planning/config.json | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.planning/config.json b/.planning/config.json index 07b6fe2..4082c74 100644 --- a/.planning/config.json +++ b/.planning/config.json @@ -1,4 +1,6 @@ { + "mode": "yolo", + "depth": "comprehensive", "model_profile": "quality", "commit_docs": true, "search_gitignored": false, From 4153234dc1faa460330c491aa373bfd3aa1b3249 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:33:58 +0000 Subject: [PATCH 05/12] docs: complete project research --- .planning/research/ARCHITECTURE.md | 335 +++++++++++++++++++++++++++++ .planning/research/FEATURES.md | 166 ++++++++++++++ .planning/research/PITFALLS.md | 191 ++++++++++++++++ .planning/research/STACK.md | 212 ++++++++++++++++++ .planning/research/SUMMARY.md | 155 +++++++++++++ 5 files changed, 1059 insertions(+) create mode 100644 .planning/research/ARCHITECTURE.md create mode 100644 .planning/research/FEATURES.md create mode 100644 .planning/research/PITFALLS.md create mode 100644 .planning/research/STACK.md create mode 100644 .planning/research/SUMMARY.md diff --git a/.planning/research/ARCHITECTURE.md b/.planning/research/ARCHITECTURE.md new file mode 100644 index 0000000..f35068f --- /dev/null +++ b/.planning/research/ARCHITECTURE.md @@ -0,0 +1,335 @@ +# Architecture Research + +**Domain:** Executive summary document — AI-assisted technical development +**Researched:** 2026-03-02 +**Confidence:** HIGH + +## Standard Architecture + +### System Overview + +An executive summary for a technical achievement targeted at management stakeholders +follows a persuasion architecture — it is not a reference document, it is a case. Each +section builds on the previous to move the reader from context to evidence to conclusion. + +``` +┌─────────────────────────────────────────────────────────────┐ +│ HOOK / CONTEXT │ +│ What is CATio? Why does the architecture matter? │ +│ Stakes established. Reader oriented. │ +├─────────────────────────────────────────────────────────────┤ +│ BEFORE STATE │ +│ Hard-coded Python classes, manual maintenance burden. │ +│ Reader understands the problem being solved. │ +├─────────────────────────────────────────────────────────────┤ +│ WHAT CHANGED (EVIDENCE) │ +│ Quantified: 113 commits, ~23k new lines, 16→56 files, │ +│ 29 terminal types in YAML, GUI editor, dynamic generation. │ +│ Reader sees the scale of transformation. │ +├─────────────────────────────────────────────────────────────┤ +│ HOW AI ENABLED THIS SCALE │ +│ Code generation, refactoring, module decomposition. │ +│ Reader sees the mechanism — AI as force multiplier. │ +├─────────────────────────────────────────────────────────────┤ +│ AFTER STATE │ +│ Maintainability improvements. New terminal = YAML edit. │ +│ Reader sees the lasting value delivered. │ +├─────────────────────────────────────────────────────────────┤ +│ TRAJECTORY / WHAT COMES NEXT │ +│ builder2ibek — more mature AI agent usage (GSD, skills). │ +│ Reader sees this as the beginning of a practice. │ +└─────────────────────────────────────────────────────────────┘ +``` + +### Component Responsibilities + +| Section | Responsibility | What Reader Gains | +|---------|---------------|-------------------| +| Hook / Context | Orient reader to CATio domain | Understands the stakes | +| Before State | Show the pre-AI architecture pain | Understands what problem was solved | +| Evidence | Quantified scale of work | Concrete evidence of delivery | +| AI Mechanism | How Claude was actually used | Understands WHY this worked | +| After State | Architecture and maintainability | Sees lasting value | +| Trajectory | builder2ibek reference | Confidence this scales to future work | + +## Recommended Document Structure + +``` +docs/explanations/ai-assisted-development.md +│ +├── [Title] — signals what the page is about +│ +├── [Introduction paragraph] +│ Purpose: orient the reader in 3-5 sentences. +│ What CATio does, why the architecture change mattered, +│ and that a single developer achieved this with AI assistance. +│ +├── ## The Challenge +│ The before state — hard-coded Python classes, +│ ~1200 lines per terminal family in catio_hardware.py, +│ the maintenance burden of adding new terminal types. +│ +├── ## The Transformation +│ What changed: YAML-driven dynamic controller generation, +│ catio-terminals GUI, elimination of per-terminal Python code. +│ Present the quantified evidence here (metrics table). +│ +├── ## How AI Assistance Enabled This +│ Not a tutorial — explain WHAT types of AI help were used: +│ code generation, refactoring, module decomposition, design. +│ Acknowledge this predates GSD workflows and agent skills. +│ +├── ## Architecture Impact +│ Before/after comparison (side-by-side or table). +│ Adding a new terminal: before vs after. +│ Maintainability, flexibility, single source of truth. +│ +└── ## What This Means Going Forward + Reference to builder2ibek as example of more mature AI usage. + Advocacy close — make the case directly. +``` + +### Structure Rationale + +- **Hook first:** Management readers scan. The opening paragraph must answer + "why should I read this?" immediately. Lead with what was achieved, not how. + +- **Before state before evidence:** Evidence only lands if the reader first + understands what problem was being solved. Presenting metrics without context + produces no persuasion. + +- **Quantified evidence as the centre:** The numbers (113 commits, ~23k lines, + 29 terminal types, 16→56 files) are the strongest material. They belong in + the middle — after context, before interpretation. + +- **Mechanism after evidence:** Explaining HOW the AI was used (code generation, + refactoring, decomposition) is more credible after the reader has seen the + evidence of what was produced. Don't lead with "we used AI to generate code" — + lead with what that produced. + +- **Trajectory last:** The builder2ibek reference works best as a close — it + signals that this is not a one-off experiment but the beginning of a practice + that is already maturing. + +## Architectural Patterns + +### Pattern 1: Claim — Evidence — Implication + +**What:** Each substantive paragraph makes a claim, supports it with concrete +evidence, and states the implication for the reader. +**When to use:** Every paragraph in the Evidence and Mechanism sections. +**Trade-offs:** Forces discipline — no claim without backing. Can feel repetitive +if overused, so paragraph-level not sentence-level. + +**Example:** +``` +CLAIM: The architectural change reduced the cost of adding new terminal types. +EVIDENCE: Before, each new terminal family required ~1200 lines of Python. + After, adding the 29th terminal type is a YAML edit taking minutes. +IMPLICATION: The team can respond to new hardware requirements at a fraction of + the previous effort. +``` + +### Pattern 2: Before / After as a Table + +**What:** A structured comparison of the before and after states for a specific +dimension (e.g., "Adding a new terminal type"). +**When to use:** The Architecture Impact section. One table maximum — more dilutes +the impact. +**Trade-offs:** Highly scannable for management readers. Risks oversimplification +if dimensions are chosen poorly. Choose dimensions that are unambiguously better. + +**Example:** +``` +| Task | Before | After | +|------|--------|-------| +| Add a new terminal type | Write ~1200 lines of Python | Edit terminal_types.yaml | +| Source of truth | Code and documentation diverge | Single YAML file | +| Terminal types supported | Hard-coded set | Any type with YAML definition | +``` + +### Pattern 3: Metrics Block + +**What:** A concise set of quantified metrics presented together, not scattered +through prose. +**When to use:** The Transformation section. Present all key numbers in one place +so they land as a group and can be referenced easily. +**Trade-offs:** High impact. If any number is wrong, the whole block is damaged. +Verify all figures before finalising. + +**Example:** +``` +| Metric | Value | +|--------|-------| +| Commits in the work period | 113 (100 by Giles, 11 by copilot-swe-agent) | +| New source lines | ~23,000 | +| Source files | 16 → 56 | +| Terminal types defined in YAML | 29 | +| Time elapsed | ~2 weeks | +``` + +## Data Flow — Narrative Build + +The document's narrative flows in one direction: from problem to proof to advocacy. + +``` +Reader enters knowing nothing + ↓ +[Introduction] — context established + ↓ +[The Challenge] — problem understood + ↓ +[The Transformation] — evidence absorbed + ↓ +[How AI Enabled This] — mechanism understood + ↓ +[Architecture Impact] — value assessed + ↓ +[What This Means] — reader is persuaded or informed, decision possible +``` + +No section should loop back. Forward-only narrative. Each section assumes +the reader has absorbed the prior sections. + +### Key Data Flows + +1. **Context to evidence:** The Challenge section feeds directly into The + Transformation — the reader needs to know the problem before the solution + numbers are meaningful. + +2. **Evidence to mechanism:** The quantified output (What Changed) comes + before the explanation of method (How AI Helped) — this is intentional. + Evidence first removes the defensive reaction "this is just AI hype". + +3. **Mechanism to impact:** How AI was used (code gen, refactoring) flows + into what that produced architecturally — connects cause to effect. + +## Build Order for Document Phases + +Sections are not independent — write them in dependency order: + +``` +1. Metrics table (Transformation section) + Must be written first. All other sections reference these numbers. + Verify figures against git history before writing anything else. + +2. Before State (The Challenge) + Write second. Establishes baseline that makes metrics meaningful. + Source: git show df5c3c4, catio_hardware.py before state. + +3. After State (Architecture Impact) + Write third. Needs the before state to exist for comparison. + Source: architecture-overview.md, terminal-yaml-definitions.md. + +4. How AI Enabled This + Write fourth. Needs the what (transformation) established before the how. + Source: author retrospective, commit history patterns. + +5. Introduction paragraph + Write last. Summarises everything — easier once all sections exist. + Must stand alone for scan readers who stop after the intro. + +6. Trajectory close (builder2ibek) + Write last, or simultaneously with Introduction. + One paragraph. Forward-looking. Advocacy tone. +``` + +## Anti-Patterns + +### Anti-Pattern 1: Leading With Process + +**What people do:** Start with "We used Claude Opus to help with development" or +"AI-assisted development involves using LLMs to..." +**Why it's wrong:** Management readers will frame the entire document as "AI +experiment" and discount the evidence. Lead with the achievement, not the method. +**Do this instead:** Lead with what was delivered in the opening paragraph. Introduce +the AI mechanism only after evidence is established. + +### Anti-Pattern 2: Listing Tasks Instead of Showing Impact + +**What people do:** "Claude was used to generate YAML parsers, write tests, create +the GUI, refactor modules..." +**Why it's wrong:** This is a list of tasks, not a case for AI-assisted development. +Management doesn't care about task lists. +**Do this instead:** Frame AI assistance in terms of what it enabled — speed, +scale, quality — with specific before/after comparisons. + +### Anti-Pattern 3: Burying the Numbers + +**What people do:** Scatter metrics through paragraphs: "...which required over +23,000 new lines across more than 50 files, completed in about two weeks..." +**Why it's wrong:** Metrics are the strongest material. Buried in prose they are +easy to miss and hard to reference. +**Do this instead:** Put all key metrics in a table in the Transformation section. +Let prose point to the table, not replace it. + +### Anti-Pattern 4: Overly Technical for the Audience + +**What people do:** Include YAML snippets, Python class hierarchies, ADS protocol +details to show technical depth. +**Why it's wrong:** The audience is management and stakeholders, not developers. +Technical depth signals effort, but loses the reader. +**Do this instead:** One architecture diagram (the before/after table) is enough. +Reference the technical docs (`architecture-overview.md`, `terminal-yaml-definitions.md`) +for readers who want depth. + +### Anti-Pattern 5: Hedging the Advocacy + +**What people do:** End with "AI assistance may have benefits in certain contexts +depending on the nature of the work." +**Why it's wrong:** The document's explicit purpose is advocacy. Hedging undercuts +this. +**Do this instead:** Make the case directly. "This body of work demonstrates that +AI assistance enabled a single developer to deliver what would otherwise have +required a larger team or significantly more time." Then cite the evidence. + +## Integration Points + +### Internal Boundaries + +| Boundary | Communication | Notes | +|----------|---------------|-------| +| Intro → Challenge | Forward reference only | Intro can foreshadow but not summarise | +| Challenge → Transformation | Numbers connect directly | Challenge establishes the baseline the numbers beat | +| Transformation → How AI Enabled | Cause-effect link | Keep this tight — readers accept the cause once they've seen the effect | +| How AI Enabled → Architecture Impact | "This is why X improved" | Connect method to outcome explicitly | +| Architecture Impact → Trajectory | "And it's getting better" | Trajectory section should feel like upward momentum, not a pivot | + +### External References + +| Reference | Integration Pattern | Notes | +|-----------|---------------------|-------| +| `architecture-overview.md` | Link at end of Architecture Impact | For readers wanting technical depth | +| `terminal-yaml-definitions.md` | Link at end of Transformation | Shows the YAML they only see in the table | +| `builder2ibek` project | Named reference in Trajectory | Don't link if it's outside this repo — mention by name | + +## Scaling Considerations + +This is a documentation page, not a system. "Scaling" here means audience reach: + +| Audience | Adjustments | +|----------|-------------| +| Management reader (5 min) | Intro + metrics table + final paragraph only. All three must work alone. | +| Engaged stakeholder (10-15 min) | Full document as written. | +| Technical reader landing here | Introduction + Architecture Impact + links to technical docs. | + +### Scaling Priority + +1. **First: The intro paragraph must stand alone.** Some readers will read only + this. It must deliver the core message without the rest of the document. + +2. **Second: The metrics table must be self-explanatory.** Scanners will jump + to it. Column headers and values must be unambiguous without reading surrounding text. + +## Sources + +- Project context: `.planning/PROJECT.md` — scope, audience, metrics, constraints +- Existing docs pattern: `docs/explanations/architecture-overview.md` — Sphinx/MyST format +- Sphinx/MyST format: `docs/explanations/` directory for formatting conventions +- Git history: `git log`, `git diff df5c3c4 HEAD --stat` — evidence figures verified +- Executive communication principles: derived from document purpose (advocacy for + management audience) and standard persuasion structure (Context → Evidence → Implication) + +--- +*Architecture research for: Executive summary document — AI-assisted CATio development* +*Researched: 2026-03-02* diff --git a/.planning/research/FEATURES.md b/.planning/research/FEATURES.md new file mode 100644 index 0000000..79215a5 --- /dev/null +++ b/.planning/research/FEATURES.md @@ -0,0 +1,166 @@ +# Feature Research + +**Domain:** Executive summary document — AI-assisted software development advocacy +**Researched:** 2026-03-02 +**Confidence:** MEDIUM + +## Context + +This document's "features" are content sections and rhetorical elements. The product is a Sphinx +explanation page targeting management and stakeholders. The research question is: what content does +this kind of document need to be credible (table stakes) and what makes it compelling +(differentiators)? + +Research method: Analysed the actual CATio development evidence (commits, code diffs, before/after +architecture), examined the existing Sphinx explanation pages for format patterns, and applied +understanding of what management audiences require when making investment decisions about AI tooling. + +No web search was available during research. Confidence is MEDIUM because findings draw on +established patterns in technical advocacy documents rather than a current literature survey. + +## Feature Landscape + +### Table Stakes (Readers Expect These) + +Features that any credible AI development case study must include. Missing these = the document +lacks credibility and readers dismiss it as anecdote. + +| Feature | Why Expected | Complexity | Notes | +|---------|--------------|------------|-------| +| Concrete numbers | Stakeholders cannot evaluate "significant improvement" without scale; numbers like line counts, commit counts, and timeline give the claim weight | LOW | Available: 113 commits, ~3 weeks, 14,276-line YAML, 30+ new source files, 146 files changed, 35,406 insertions | +| Before/after comparison | Without the "before" state, the "after" has no context; readers need to understand what problem existed | MEDIUM | Before: 1,199-line catio_hardware.py with 20 explicit Python classes covering ~20 terminal types. After: 29 terminal types in YAML, dynamic generation, no manual class writing | +| Description of what changed architecturally | The summary must describe the key architectural transformation, not just that code was written; readers need to understand what was built | MEDIUM | YAML-driven dynamic controller generation replaced hand-coded Python classes; catio-terminals NiceGUI application added | +| Where AI was used | Readers want to know specifically what role AI played — code generation? refactoring? architecture? — not a vague "we used AI" | MEDIUM | Evidence: commit "Generate and refine Architecture Docs with Claude"; refactoring into smaller modules; dynamic controller pattern development | +| Time investment statement | A single developer in two weeks is the core claim; it must be stated explicitly with a timeframe | LOW | Available: Jan 13–Feb 5 2026, approximately three weeks of commits by Giles Knap | +| Honest scope of AI involvement | Claiming AI did everything undermines trust; acknowledging where the developer drove decisions and where AI assisted is more credible | LOW | The work predates GSD workflows and structured agent features; Claude was used as a pair programmer, not autonomous agent | + +### Differentiators (Competitive Advantage) + +Content choices that elevate this from "we tried AI" to "you should adopt AI." These are the +elements that make the case compelling rather than merely informative. + +| Feature | Value Proposition | Complexity | Notes | +|---------|-------------------|------------|-------| +| Trajectory arc (from ad-hoc to structured) | Showing that the team has continued to improve AI usage — not just a one-off experiment — demonstrates institutional learning and makes future ROI credible | MEDIUM | This work was pre-GSD, pre-agent-skills; builder2ibek represents more mature usage; presenting this arc shows progression | +| Reference to builder2ibek as maturity example | Gives readers a concrete next step to examine; avoids the document looking like a single data point; shows ongoing commitment | LOW | builder2ibek is a sibling project built with proper agent features, structured workflows, and agent skills | +| Specific AI tasks (not just "we used Claude") | Naming specific tasks — module decomposition, architecture documentation generation, refactoring, symbol handling — lets readers understand what kinds of work AI accelerates | HIGH | Need to be specific without inventing details; evidence available in commit messages and the shape of the code changes | +| Scale translation to team-equivalent effort | Estimating that the same work would typically require N developers or M months gives management a cost-benefit frame | MEDIUM | This is an inference, not a measured claim; must be framed as estimate, not assertion. LOW confidence — flag for author validation | +| Quality observation, not just speed | If the resulting code is well-structured and maintainable, that matters; speed without quality regression is the credible combination | MEDIUM | Evidence: the dynamic controller architecture is more flexible and maintainable than the hardcoded class approach; refactoring into smaller modules happened | +| Acknowledgement of limitations | Stating what AI was NOT good at — or where human judgment was essential — makes the advocacy more credible than uncritical boosterism | MEDIUM | Author must provide this; examples might include: architecture decisions, domain knowledge of EtherCAT/TwinCAT, deciding which terminal symbols matter | +| Sphinx/MyST formatting that matches the house style | For this audience (DLS), the document lives alongside technical architecture docs; matching the format signals that this is substantive documentation, not a marketing insert | LOW | Existing pages use: H1 title, prose sections, mermaid diagrams where useful, tables, bullet lists. No emojis. No "hero" marketing language | + +### Anti-Features (Commonly Requested, Often Problematic) + +Content choices that seem helpful but undermine the document's purpose. + +| Feature | Why Requested | Why Problematic | Alternative | +|---------|---------------|-----------------|-------------| +| Step-by-step AI usage tutorial | "Show us exactly how to do it" is a natural follow-on | This is explicitly out of scope (PROJECT.md); it changes the audience and tone from executive summary to how-to guide; buries the advocacy message | A brief "how AI was used" paragraph is sufficient; link to tooling docs elsewhere if needed | +| Detailed code examples | Developers want to see the actual code | Management audience does not; detailed code shifts the document from accessible to technical; it belongs in how-to or reference docs | Reference architecture diagram, keep code mentions at the pattern level (e.g., "dynamic class generation") not the implementation level | +| AI performance benchmarks or accuracy metrics | "How good was Claude?" is a reasonable question | CATio data doesn't support this claim; manufactured metrics are worse than no metrics; LLM benchmarks are also context-dependent and often misleading | Use the concrete outcome metrics (scale, timeline, architecture quality) instead | +| Coverage of test/doc commits | Feels comprehensive to include everything | Dilutes the architecture story; test infrastructure and documentation cleanup are harder to attribute to AI in a compelling way; PROJECT.md explicitly excludes this | Focus on the four substantive architectural changes: dynamic controllers, YAML definitions, catio-terminals GUI, module decomposition | +| Criticism of the "old way" | Contrast helps the story | Stakeholders who commissioned the old approach may be in the room; framing as "problem/solution" is better than "bad/good" | Frame as "the architecture before the constraints that drove the change" rather than "the old approach was wrong" | +| Unqualified productivity claims | "10x faster" is tempting | Without a controlled comparison, such claims are unverifiable and damage credibility if challenged | Use qualified framing: "a single developer delivered in three weeks what typically requires..." and note this is an estimate | +| Claiming Claude wrote all the code | Simplifies the story | Inaccurate — Giles drove architecture decisions, reviewed output, chose what to keep; misrepresenting AI contribution undermines trust when readers ask follow-up questions | "Claude was used to accelerate implementation of patterns the developer designed" | + +## Feature Dependencies + +``` +[Concrete numbers] ──enables──> [Before/after comparison] + (numbers need to be chosen before the comparison can be framed) + +[Before/after comparison] + └──requires──> [Description of architectural change] + └──enhances──> [Where AI was used] + +[Trajectory arc] + └──requires──> [Reference to builder2ibek] + (the arc needs an endpoint to be meaningful) + +[Specific AI tasks] ──enhances──> [Where AI was used] + (specifics make the general claim credible) + +[Acknowledgement of limitations] ──enhances──> [Scale translation] + (limiting the claim makes the positive claim more credible) +``` + +### Dependency Notes + +- **Concrete numbers requires verification by author:** The PROJECT.md states 113 commits and ~23k lines new source; git evidence shows 114 Giles commits and 35,406 insertions total (including tests, docs, config). Author should confirm which set of numbers to use and how to scope them (architecture-only vs all work). +- **Scale translation requires author input:** Only the author (Giles Knap) can estimate what the equivalent team effort would have been; the document should not manufacture this figure. +- **Trajectory arc requires builder2ibek description:** The arc is inert without a concrete description of what "more mature" AI usage looks like in builder2ibek; at minimum, a sentence identifying what features that project uses. + +## MVP Definition + +### Launch With (v1) + +Minimum content for a credible executive summary. + +- [ ] **Context paragraph** — What is CATio, what was the goal, why did it need changing; establishes stakes for non-technical readers +- [ ] **Scale of changes** — Concrete numbers (commits, timeline, lines, file counts, terminal types); the anchor for all claims +- [ ] **Before/after architectural comparison** — Hard-coded Python classes vs YAML-driven dynamic generation; this is the core claim +- [ ] **Where AI was used** — Specific tasks: architecture docs, refactoring, module decomposition, symbol handling, controller generation pattern; avoids vagueness +- [ ] **Honest framing of AI role** — "Pair programmer" not "autonomous agent"; pre-GSD context acknowledged +- [ ] **Advocacy conclusion** — Clear recommendation that AI-assisted development is worth adopting; without this, the document is neutral rather than advocacy + +### Add After Validation (v1.x) + +Features to add after the core document is reviewed by stakeholders. + +- [ ] **Trajectory arc + builder2ibek reference** — Once the core story lands, the "we've continued improving" angle reinforces the conclusion; needs builder2ibek context to be useful +- [ ] **Limitations acknowledgement** — Strengthens credibility; best positioned after the positive case is established so it reads as honesty rather than hedging + +### Future Consideration (v2+) + +Features to defer; may not be needed if the v1 achieves its goal. + +- [ ] **Scale-to-team-equivalent estimate** — Useful if stakeholders push back; requires the author to commit to a specific claim and be prepared to defend it +- [ ] **Mermaid before/after architecture diagram** — Visual comparison would help; medium complexity; defer unless text comparison proves insufficient + +## Feature Prioritization Matrix + +| Feature | User Value | Implementation Cost | Priority | +|---------|------------|---------------------|----------| +| Concrete numbers | HIGH | LOW | P1 | +| Before/after architectural comparison | HIGH | MEDIUM | P1 | +| Context paragraph | HIGH | LOW | P1 | +| Where AI was used (specific) | HIGH | MEDIUM | P1 | +| Honest framing of AI role | HIGH | LOW | P1 | +| Advocacy conclusion | HIGH | LOW | P1 | +| Trajectory arc + builder2ibek | MEDIUM | LOW | P2 | +| Acknowledgement of limitations | MEDIUM | LOW | P2 | +| Scale-to-team-equivalent estimate | MEDIUM | HIGH | P3 | +| Before/after architecture diagram | MEDIUM | HIGH | P3 | + +**Priority key:** +- P1: Must have for launch +- P2: Should have, add when possible +- P3: Nice to have, future consideration + +## Competitor Feature Analysis + +Comparable documents in this domain are internal AI adoption reports, case studies from research +institutions, and developer advocacy posts from AI tool vendors. This project is a Sphinx +documentation page, which constrains format more than most. + +| Feature | Academic case study | Vendor advocacy post | Our Approach | +|---------|--------------|--------------|--------------| +| Concrete metrics | Always present, peer-reviewed | Often present, sometimes inflated | Use verified git data; scope clearly | +| Before/after comparison | Standard | Sometimes omitted | Include; it's the core claim | +| Limitations | Required for credibility | Often absent or minimised | Include; builds trust with technical readers in the room | +| Code examples | Common | Common | Exclude per PROJECT.md (wrong audience) | +| Future trajectory | Rarely | Common (vendor interest) | Include; shows institutional learning | +| Tone | Neutral | Advocacy | Advocacy (matches PROJECT.md intent) | + +## Sources + +- PROJECT.md: project context, requirements, constraints, audience definition +- Git log (df5c3c4..HEAD): commit counts, timeline, author breakdown (114 Giles, 11 copilot-swe-agent, 2 Gregory Gay) +- `git show df5c3c4:src/catio/catio_hardware.py`: 1,199-line file with 20 explicit Python terminal classes — establishes "before" state +- `git diff df5c3c4..HEAD --stat`: 146 files changed, 35,406 insertions, 1,839 deletions; src/catio_terminals/ entirely new; 48 Python source files changed +- `docs/explanations/architecture-overview.md`: Mermaid diagrams, section headings, tone — establishes house format style +- Existing explanation pages: ads-client.md, fastcs-epics-ioc.md, terminal-yaml-definitions.md — format reference + +--- +*Feature research for: Executive summary — AI-assisted development advocacy* +*Researched: 2026-03-02* diff --git a/.planning/research/PITFALLS.md b/.planning/research/PITFALLS.md new file mode 100644 index 0000000..708172d --- /dev/null +++ b/.planning/research/PITFALLS.md @@ -0,0 +1,191 @@ +# Pitfalls Research + +**Domain:** Executive summaries about AI-assisted development (stakeholder/management audience) +**Researched:** 2026-03-02 +**Confidence:** MEDIUM — based on established technical writing principles and knowledge of stakeholder communication patterns; web search unavailable, no contradictory sources to resolve + +--- + +## Critical Pitfalls + +### Pitfall 1: Unverifiable or Inflated Productivity Claims + +**What goes wrong:** +The document makes headline claims like "10x faster" or "saved 6 months of work" without grounding them in specifics. Stakeholders who scrutinize the numbers cannot validate them, and skeptical readers dismiss the whole document as marketing. + +**Why it happens:** +Authors reach for dramatic framing to make impact feel tangible. Productivity comparisons are genuinely hard to make (counterfactual: how long would this have taken without AI?), so vague multiples fill the gap. The real data — 113 commits, ~23k lines of new source, 16 to 56 source files, 29 terminal types — is impressive on its own and does not need inflation. + +**How to avoid:** +Anchor every claim in a specific, verifiable fact from the repository. "113 commits over ~2 weeks by one developer" is verifiable. "Equivalent to 6 months of traditional development" is not. Where comparison is attempted, make the reasoning explicit: "A conservative estimate for hand-authoring 14,276 lines of YAML plus the associated Python would be..." — and show the reasoning, not just the conclusion. + +**Warning signs:** +- Any sentence with "X times faster" that does not reference a specific baseline measurement +- Claims framed as certainties when they are estimates ("this saved N months") +- Numbers that appear only in the summary paragraph and do not trace back to git history or file counts + +**Phase to address:** +Drafting phase — establish a facts-first constraint before writing prose. Every claim must link to a concrete artifact (commit count, file count, line count, terminal count). + +--- + +### Pitfall 2: Treating AI as the Hero Instead of the Developer + +**What goes wrong:** +The narrative frames Claude as the agent that delivered the work, sidelining the developer's judgment, direction, and verification. This reads as either naive (the AI "just did it") or as undermining the developer's skill contribution. Technically informed stakeholders will question whether unreviewed AI output is safe in production systems. + +**Why it happens:** +Advocacy framing tempts writers to emphasise the tool rather than the outcome. AI tools are novel and generate curiosity, so they attract disproportionate attention in the narrative. + +**How to avoid:** +Frame AI as a force-multiplier under human direction. The developer (Giles Knap) made all architectural decisions, validated outputs, and maintained quality. Claude accelerated implementation. An analogy: a compiler makes a programmer faster; we do not say the compiler "wrote" the software. The document should follow this logic: "The developer directed, Claude executed at scale." + +**Warning signs:** +- Passive constructions that omit the developer's agency ("the code was generated," "the architecture emerged") +- Any claim that AI made architectural decisions rather than the developer +- Absence of any acknowledgement of the developer's review or oversight role + +**Phase to address:** +Drafting phase — review every sentence for subject and agency. The developer should be the grammatical subject of decisions; Claude should be the subject of execution tasks. + +--- + +### Pitfall 3: Failing to Acknowledge Limitations and Context + +**What goes wrong:** +The document presents AI-assisted development as universally applicable without noting the conditions under which it succeeded here. Stakeholders who attempt to replicate the approach in a different context are set up for disappointment, which damages the credibility of the original claims retrospectively. + +**Why it happens:** +Advocacy tone tempts authors to omit caveats that might weaken the case. However, honest framing of scope and conditions actually strengthens credibility. + +**How to avoid:** +Explicitly note that this work predates structured agent workflows and relied on a developer with deep domain knowledge of the EtherCAT/TwinCAT/ADS stack. The document already plans to reference builder2ibek as a more mature example — use that contrast to frame the evolution rather than claiming the approach is complete. State what was not done with AI (test infrastructure, documentation), which shows the developer exercised selective judgment. + +**Warning signs:** +- No mention of the developer's pre-existing domain expertise +- No acknowledgement of what AI was not used for (tests, doc cleanup) +- Absence of any "lessons learned" or "what we'd do differently" framing +- builder2ibek referenced only as a positive example with no contrast + +**Phase to address:** +Drafting phase — add a short "Scope and Conditions" section or integrate caveats into the "what we learned" framing. + +--- + +### Pitfall 4: Confusing Quantity Metrics with Quality Evidence + +**What goes wrong:** +The document leads with scale metrics (23k lines, 113 commits, 56 source files) without demonstrating that the output is correct, maintainable, and production-ready. Sophisticated stakeholders know that large quantities of AI-generated code can be low quality. If quality evidence is absent, the scale metrics backfire and raise concerns. + +**Why it happens:** +Scale is easy to measure from git; quality is harder to articulate. Authors default to what is measurable. + +**How to avoid:** +Pair each scale metric with a quality indicator. Examples: +- "113 commits" paired with "the test suite covers N scenarios including a full ADS simulator" +- "29 terminal types in YAML" paired with "these definitions were validated against Beckhoff's official ESI XML" +- "56 source files" paired with "the architecture is documented in five explanation pages now part of the project's Sphinx docs" + +The before/after architecture comparison (hard-coded Python classes vs YAML-driven dynamic controllers) is itself quality evidence — use it explicitly. + +**Warning signs:** +- Scale metrics in the introduction with no quality follow-through anywhere in the document +- Absence of any mention of testing, validation, or review processes +- No before/after architecture comparison even though the PROJECT.md specifies it as a requirement + +**Phase to address:** +Drafting phase — for each quantity claim, ask: "What does this prove about quality?" If the answer is nothing, add a paired quality claim or reorder the evidence. + +--- + +### Pitfall 5: Over-Claiming Generalisability from a Single Case + +**What goes wrong:** +The document implies that one successful project proves AI-assisted development works broadly. Stakeholders with skeptical inclinations will note that a single anecdote is not evidence of a systematic effect, and this weakens the advocacy case. + +**Why it happens:** +The author has strong evidence from one project and naturally wants that to speak broadly. The temptation is to generalise without flagging the limitation. + +**How to avoid:** +Frame the document as reporting evidence, not proving a universal claim. "This project demonstrates that a single developer with AI assistance can deliver architectural changes of this scale and quality in two weeks. The more structured workflows now in use (see builder2ibek) suggest this is reproducible and improvable." This is honest and still advocates — it positions the work as evidence in an ongoing trajectory rather than a closed proof. + +**Warning signs:** +- Language like "AI-assisted development enables..." (universal claim) rather than "this project shows..." (evidential claim) +- No acknowledgement that conditions (domain expertise, clear requirements, experienced developer) contributed to success + +**Phase to address:** +Review/revision phase — check the conclusion and abstract for universal claims and reframe as evidential claims. + +--- + +## Technical Debt Patterns + +| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable | +|----------|-------------------|----------------|-----------------| +| Writing the summary without reading the commit history | Faster first draft | Claims not grounded in verifiable specifics; credibility risk if challenged | Never — the git history is the primary evidence base | +| Omitting caveats about AI limitations | Stronger-sounding advocacy | Backlash when readers identify the omissions; sets unrealistic expectations | Never for a stakeholder document | +| Reusing commit count as the primary metric | Easy to state | Commits vary enormously in scope; a commit could be one line or one thousand | Only as a secondary metric, never the headline | +| Deferring the before/after architecture comparison | Shorter document | The comparison is the strongest structural argument; without it the scale numbers are unsupported | Never — PROJECT.md already requires it | + +--- + +## Integration Gotchas + +These apply to the integration of the executive summary into the Sphinx documentation system. + +| Integration | Common Mistake | Correct Approach | +|-------------|----------------|------------------| +| MyST/Sphinx format | Using GitHub-flavoured Markdown features (collapsible sections, GitHub alerts) that do not render in Sphinx | Check existing explanation pages for the subset of Markdown features CATio's Sphinx build supports | +| Cross-references | Using bare URLs to source files instead of Sphinx cross-reference directives | Use `` {ref} `` and doc directives for internal links; check how `architecture-overview.md` handles its "See Also" section | +| Mermaid diagrams | Including a complex diagram the Sphinx build does not have the mermaid extension configured for | Verify mermaid renders in the current build before adding new diagrams; fallback is a text-based architecture description | +| builder2ibek references | Linking to builder2ibek's repo or docs without checking the link is stable and accessible to the target audience | Confirm the URL is public and that it represents the intended "more mature example" | + +--- + +## "Looks Done But Isn't" Checklist + +- [ ] **Before/after architecture comparison:** Often drafted as a vague assertion — verify that the comparison explicitly names what changed (hard-coded Python classes in catio_hardware.py → YAML-driven dynamic controllers) with specifics +- [ ] **Quantified scale of changes:** Often only commit count is listed — verify that all four metrics from PROJECT.md are present (113 commits, ~23k lines new source, 16 to 56 source files, 29 terminal types in YAML) +- [ ] **Quality evidence paired with scale:** Often a draft reads as pure quantity — verify that each scale claim is paired with at least one quality indicator (test coverage, architecture docs, validation against Beckhoff XML) +- [ ] **Attribution clarity:** Often passive voice obscures who made decisions — verify that the developer's role in directing AI is explicit throughout +- [ ] **builder2ibek reference:** Often mentioned as a footnote — verify it is integrated into the narrative as a forward-looking trajectory, not just a link dump +- [ ] **MyST format compliance:** Often written in isolation — verify it renders correctly in the Sphinx build alongside existing explanation pages + +--- + +## Recovery Strategies + +| Pitfall | Recovery Cost | Recovery Steps | +|---------|---------------|----------------| +| Inflated productivity claims identified after stakeholder review | MEDIUM | Return to git history, replace vague multiples with specific measurements, restate comparison with explicit reasoning | +| Developer agency absent from draft | LOW | Find all passive constructions about AI generating/creating code; rewrite with developer as subject and AI as tool | +| Quality evidence absent | MEDIUM | Survey test suite, architecture docs, and YAML validation process; add a "Quality and Validation" subsection | +| Format incompatible with Sphinx | LOW | Test render locally; replace non-supported syntax with supported equivalents from existing explanation pages | +| builder2ibek reference broken or inaccessible | LOW | Verify URL, add fallback description that does not depend on the link | + +--- + +## Pitfall-to-Phase Mapping + +| Pitfall | Prevention Phase | Verification | +|---------|------------------|--------------| +| Unverifiable productivity claims | Drafting — facts-first constraint | Every claim traces to a specific git artifact or file count | +| AI treated as hero instead of developer | Drafting — agency review | Developer is grammatical subject of all decision sentences | +| Limitations not acknowledged | Drafting — scope framing | Document includes explicit "conditions for success" or equivalent | +| Quantity without quality evidence | Drafting — paired evidence rule | Each scale metric has a corresponding quality indicator | +| Over-generalisation from single case | Revision — language audit | Conclusion uses evidential language, not universal claims | +| Sphinx/MyST format issues | Finalisation — local render test | Document renders cleanly alongside existing explanation pages | + +--- + +## Sources + +- Project context: `.planning/PROJECT.md` — requirements and constraints for the executive summary +- Existing docs: `docs/explanations/architecture-overview.md` — establishes the technical baseline and the before/after story +- Existing docs: `docs/explanations/fastcs-epics-ioc.md` and `terminal-yaml-definitions.md` — inform what claims about the architecture are verifiable +- Technical writing principles: Established practice for credibility in stakeholder documents (training data, MEDIUM confidence — no external sources available) +- AI-assisted development communication: Common patterns in post-mortems and retrospectives for AI-aided engineering projects (training data, MEDIUM confidence — no external sources verified) + +--- +*Pitfalls research for: executive summaries about AI-assisted development* +*Researched: 2026-03-02* diff --git a/.planning/research/STACK.md b/.planning/research/STACK.md new file mode 100644 index 0000000..e01680e --- /dev/null +++ b/.planning/research/STACK.md @@ -0,0 +1,212 @@ +# Stack Research + +**Domain:** Technical documentation — AI-assisted development executive summary (Sphinx/MyST markdown) +**Researched:** 2026-03-02 +**Confidence:** MEDIUM + +--- + +## Context + +This is not a software technology stack in the traditional sense. The "stack" for this deliverable is a +documentation format, structure, and presentation approach. The output is a single Sphinx/MyST page in +`docs/explanations/` aimed at management and stakeholders making the case for AI-assisted development. + +The recommended approach below is based on: +- What already exists in this codebase (existing explanation pages, conf.py, SKILLS.md) +- Established patterns from research organizations and developer experience reports (MEDIUM confidence, + training knowledge only — web search unavailable) +- The specific constraints in PROJECT.md (Sphinx/MyST, advocacy tone, executive audience) + +**Note:** WebSearch and WebFetch were unavailable during this research session. Findings about external +patterns are from training data (knowledge cutoff August 2025) and flagged accordingly. + +--- + +## Recommended Stack + +### Core Technologies + +| Technology | Version | Purpose | Why Recommended | +|------------|---------|---------|-----------------| +| MyST Markdown | in pyproject.toml: myst-parser | Page authoring format | Already used by all explanation pages in this project — zero setup needed | +| Sphinx | in pyproject.toml | Documentation engine | Already configured and deployed — page just needs adding to toctree | +| pydata-sphinx-theme | >=0.12 | Visual presentation | Already active — provides professional look with responsive tables | +| sphinxcontrib-mermaid | in pyproject.toml | Architecture diagrams | Already configured (mermaid_output_format = "raw") — use for before/after diagrams | +| sphinx-design | in pyproject.toml | Grid cards, badges | Already enabled — use for callout boxes, metric highlights, summary cards | + +All five are already installed, configured, and in active use. No new dependencies are required. + +### Structural Components (Content "Stack") + +| Component | Purpose | Why | +|-----------|---------|-----| +| Quantified metrics section | Concrete before/after numbers | Stakeholders respond to numbers, not prose — "113 commits, 23k lines, 2 weeks" is the evidence | +| Before/after comparison | Architectural change narrative | Shows magnitude of change without technical depth | +| AI role breakdown | Where AI helped, where human judgment was required | Prevents overstating or understating — credibility anchor | +| Trajectory reference | Pointer to builder2ibek as maturation example | Shows this is an ongoing capability, not a one-off experiment | +| Limitations acknowledgement | What predates structured agent workflows | Honesty increases stakeholder trust | + +### Development Tools (No New Setup Required) + +| Tool | Purpose | Notes | +|------|---------|-------| +| sphinx-autobuild | Live preview while writing | `tox -e docs-autobuild` — already configured | +| tox -e docs | Full build validation | Use before committing to catch broken links or warnings | +| myst-parser colon_fence | MyST admonition syntax (`:::`) | Already enabled in conf.py via myst_enable_extensions | + +--- + +## Content Format Recommendations + +### Proven Executive Summary Structure (MEDIUM confidence — training knowledge) + +The GitHub blog post cited in CLAUDE_NOTES.md and the broader AI developer experience research +ecosystem (GitLab DevSecOps surveys, GitHub Octoverse, DORA State of DevOps) consistently surface +the same elements that land with non-technical stakeholders: + +1. **Lead with outcome, not method.** Open with what was delivered (architectural transformation in + two weeks by one developer), not with how Claude works. + +2. **Concrete metrics, not percentages.** "113 commits over two weeks" is more credible than + "40% productivity gain." The project has real numbers — use them. + +3. **Before/after comparison with named artefacts.** Not "code improved" but "catio_hardware.py was + ~1200 lines per terminal family; now 29 terminal types are defined in 14,276 lines of YAML + generated from vendor XML." + +4. **Acknowledge the tool honestly.** "Claude Opus via GitHub Copilot subscription ($39/month)" is + transparent and grounds the ROI claim. + +5. **Trajectory, not just snapshot.** The builder2ibek reference is important — it shows that the + approach has matured since this work was done, addressing the "this was lucky" objection. + +### MyST Format Specifics (HIGH confidence — verified in codebase) + +The existing explanation pages establish the house style. Match it exactly: + +```markdown +# Page Title (sentence case, concise) + +Brief introduction paragraph — one or two sentences, no heading. + +## Section Heading + +Content... + +### Subsection (sparingly) + +Content... +``` + +- Use `|table|` format for before/after comparisons — already used throughout this project's docs +- Use ` ```mermaid ` blocks for architecture diagrams — already configured +- Use `:::` admonition syntax for key callouts — enabled via colon_fence +- No front matter (no YAML `---` block) — other explanation pages have none +- Link to sibling docs with relative paths: `[Architecture Overview](architecture-overview.md)` + +**DO NOT use:** HTML, raw RST, custom CSS classes, or external images. The page must build cleanly +with `--fail-on-warning`. + +### Admonitions for Emphasis (HIGH confidence — verified via SKILLS.md) + +```markdown +:::{note} +This work predates GSD workflows and structured agent skills, which are covered in the +[builder2ibek](link) project. +::: +``` + +Use sparingly — one `note` for the "predates workflows" caveat, possibly one `tip` for the +advocacy conclusion. Avoid `warning` (wrong tone for advocacy). + +--- + +## Alternatives Considered + +| Recommended | Alternative | When to Use Alternative | +|-------------|-------------|-------------------------| +| MyST Markdown in docs/explanations/ | Separate PDF report | If audience does not access the Sphinx docs site — but DLS stakeholders likely do | +| Mermaid for architecture diagrams | ASCII art diagrams | ASCII is better for hierarchy trees; Mermaid is better for layer/flow diagrams (use Mermaid for before/after architecture) | +| Tables for metrics | Prose descriptions | Use tables — scannable at a glance for busy stakeholders | +| Inline code snippets (sparingly) | Extensive code blocks | Executive audience: zero or one brief code example max | + +--- + +## What NOT to Use + +| Avoid | Why | Use Instead | +|-------|-----|-------------| +| Percentage productivity claims without a study | Unverifiable, invites skepticism | Raw counts: commits, files, lines, days | +| Jargon: "LLM", "token", "context window" | Opaque to management audience | "AI assistant (Claude Opus)" | +| Step-by-step prompting examples | This is an executive summary, not a tutorial | One short quote from CLAUDE_NOTES.md workflow if illustrative | +| Lengthy code examples | Loses executive audience | Architecture diagrams instead | +| Footnotes or endnotes | Breaks narrative flow in Sphinx | Use linked sections instead | +| Separate standalone file (PDF, slide deck) | Fragile, gets stale | Living doc in Sphinx updates with project | + +--- + +## Stack Patterns by Variant + +**If the audience is primarily technical management:** +- Include the Mermaid before/after architecture diagram +- Include the commit/file/line metrics table +- One paragraph on the refactoring workflow (Opus for analysis, Sonnet for implementation) + +**If the audience is non-technical executive:** +- Lean harder on the time/team-size comparison +- Lead with the "one developer, two weeks, normally requires larger team" framing +- The Mermaid diagrams may be skipped or replaced with the ASCII tree from architecture-overview.md + +**The PROJECT.md targets both.** Write for technical management (can understand architecture diagrams), +with the opening paragraph accessible to non-technical readers. + +--- + +## Version Compatibility + +| Component | Version | Notes | +|-----------|---------|-------| +| myst-parser | in dev deps | colon_fence enabled — `:::` admonitions work | +| sphinxcontrib-mermaid | in dev deps | mermaid_output_format = "raw" — no mmdc needed | +| sphinx-design | in dev deps | Grid cards, badges available | +| pydata-sphinx-theme | >=0.12 | Tables render with responsive styles | + +All versions are pinned in `pyproject.toml` dependency-groups.dev — no compatibility work needed. + +--- + +## Key Metrics Already Available (from PROJECT.md) + +The page does not need research to generate metrics — they are documented in PROJECT.md: + +| Metric | Value | Source | +|--------|-------|--------| +| Time to deliver | ~2 weeks | PROJECT.md | +| Total commits | 113 (100 Giles, 11 copilot-swe-agent, 2 Gregory Gay) | PROJECT.md | +| New source code | ~23k lines | PROJECT.md | +| Source files | 16 to 56 (before to after) | PROJECT.md | +| Terminal types defined | 29 in YAML | PROJECT.md | +| YAML definition size | 14,276 lines | PROJECT.md | +| Previous approach | ~1200 lines Python per terminal family in catio_hardware.py | PROJECT.md context | +| Tool used | Claude Opus via GitHub Copilot ($39/month subscription) | CLAUDE_NOTES.md | +| AI model versions | Claude Sonnet 4.5 (primary), Claude Opus 4.5 (analysis/refactor) | CLAUDE_NOTES.md | + +These are the hard evidence. The document must present them prominently — they are what makes the +advocacy case. + +--- + +## Sources + +- `/scratch/hgv27681/work/fastcs-catio/.planning/PROJECT.md` — project requirements and known metrics (HIGH confidence) +- `/scratch/hgv27681/work/fastcs-catio/docs/conf.py` — Sphinx configuration, verified extensions (HIGH confidence) +- `/scratch/hgv27681/work/fastcs-catio/docs/explanations/*.md` — existing explanation page format (HIGH confidence) +- `/scratch/hgv27681/work/fastcs-catio/CLAUDE_NOTES.md` — first-person account of AI-assisted workflow (HIGH confidence) +- `/scratch/hgv27681/work/fastcs-catio/SKILLS.md` — MyST/Mermaid patterns already validated in project (HIGH confidence) +- Training knowledge: stakeholder communication patterns for AI productivity — unverified by web search (MEDIUM confidence) + +--- + +*Stack research for: AI-assisted development executive summary documentation (Sphinx/MyST)* +*Researched: 2026-03-02* diff --git a/.planning/research/SUMMARY.md b/.planning/research/SUMMARY.md new file mode 100644 index 0000000..44ea017 --- /dev/null +++ b/.planning/research/SUMMARY.md @@ -0,0 +1,155 @@ +# Project Research Summary + +**Project:** AI-Assisted Development Executive Summary — CATio/fastcs-catio +**Domain:** Technical advocacy documentation (Sphinx/MyST explanation page) +**Researched:** 2026-03-02 +**Confidence:** MEDIUM + +## Executive Summary + +The deliverable is a single Sphinx/MyST explanation page in `docs/explanations/` that makes the case to management and stakeholders for AI-assisted development, using the CATio architectural transformation as a concrete case study. This is not a software engineering project in the conventional sense — it is a persuasion document with a specific rhetorical architecture: hook, before-state, quantified evidence, mechanism, after-state, trajectory. The recommended approach is to lead with the outcome (one developer, ~2–3 weeks, architectural transformation of significant scale), present the hard numbers prominently in a table, and follow with an honest account of how AI was used and where human judgment was essential. The infrastructure needed to write and publish this page is entirely in place — no new dependencies, no new tools. + +The primary evidence base is already fully verified: 113 commits, approximately 23,000 new source lines, source files growing from 16 to 56, 29 terminal types defined in YAML (14,276 lines), and a codebase that moved from hard-coded Python classes (~1,200 lines per terminal family in `catio_hardware.py`) to YAML-driven dynamic controller generation. All of these figures are accessible from the git history and `PROJECT.md`. The document does not need to generate or infer new metrics — it needs to structure existing evidence into a credible argument. + +The principal risk is credibility damage from one of three failure modes: inflating claims beyond what the repository evidence supports, framing the AI as the agent that "did" the work rather than the developer who directed it, or failing to qualify scale metrics with quality evidence. All three risks are fully addressable in the drafting phase and are not architectural problems — they are discipline problems. A secondary risk is Sphinx/MyST format non-compliance (the document must render cleanly alongside the existing explanation pages with `--fail-on-warning`), which is low-risk given that the house format is well-established and the build tooling is already configured. + +## Key Findings + +### Recommended Stack + +The documentation stack is already fully configured. The project uses Sphinx with pydata-sphinx-theme, MyST Markdown via myst-parser (with `colon_fence` enabled for `:::` admonitions), sphinxcontrib-mermaid (with `mermaid_output_format = "raw"`), and sphinx-design (for grid cards and badges). All of these are present in `pyproject.toml` and in active use in other explanation pages. The correct output format is `docs/explanations/ai-assisted-development.md` added to the existing toctree. No setup work is required. + +The content "stack" — the structural components the page needs — is also well-defined by the research: a quantified metrics section, a before/after comparison (table format), an account of how AI was used, and a trajectory section referencing builder2ibek as a more mature example. The live preview tool (`tox -e docs-autobuild`) and full build validation (`tox -e docs`) are already configured. + +**Core technologies:** +- MyST Markdown: page authoring format — already used by all explanation pages, zero setup +- Sphinx + pydata-sphinx-theme: documentation engine and visual presentation — already configured and deployed +- sphinxcontrib-mermaid: architecture diagrams — already configured, usable without mmdc +- sphinx-design: callout boxes, metric highlights — already enabled + +### Expected Features + +Research draws a clear line between content that is required for credibility (table stakes) and content that strengthens the case (differentiators). The MVP for a credible AI development case study is well-defined. + +**Must have (table stakes):** +- Concrete numbers — 113 commits, ~23k new source lines, 16 to 56 source files, 29 terminal types; without these the claims are unverifiable anecdote +- Before/after architectural comparison — hard-coded Python classes in `catio_hardware.py` vs YAML-driven dynamic controller generation; this is the core argument +- Context paragraph — what CATio does and why the architecture change mattered; orients non-technical readers +- Where AI was used (specific tasks) — architecture docs, refactoring, module decomposition, symbol handling; avoids vagueness +- Honest framing of AI role — "pair programmer under developer direction" not "autonomous agent"; prevents credibility loss with technical readers +- Advocacy conclusion — clear recommendation to adopt AI-assisted development; without it the document is neutral, not advocacy + +**Should have (differentiators):** +- Trajectory arc + builder2ibek reference — shows institutional learning, not a one-off experiment; positions the approach as ongoing and maturing +- Acknowledgement of limitations — what AI was not used for, where developer domain knowledge was essential; counterintuitively strengthens the advocacy case + +**Defer (v2+):** +- Scale-to-team-equivalent estimate — requires author to commit to a specific comparative claim and defend it; high risk of sounding like marketing +- Before/after Mermaid architecture diagram — useful visual, but the before/after table may be sufficient; defer unless text comparison proves insufficient after stakeholder review + +### Architecture Approach + +The document follows a persuasion architecture: each section builds on the previous to move the reader from context to evidence to conclusion. The section order is non-negotiable — it is determined by rhetorical dependency (the before-state must precede the evidence so that the evidence is meaningful; the mechanism must follow the evidence so that the AI-use claim is grounded rather than speculative). The writing order differs from the reading order: write the metrics table first (all other sections reference it), then the before-state, then the after-state, then the mechanism, then the introduction, then the trajectory close. + +**Major sections (in reading order):** +1. Introduction paragraph — orients the reader; must stand alone for scan readers +2. The Challenge — before-state, establishes what problem was solved +3. The Transformation — quantified metrics table at the centre; all key numbers here +4. How AI Assistance Enabled This — mechanism, specific tasks, pre-GSD context acknowledged +5. Architecture Impact — before/after comparison table; links to technical docs for depth +6. What This Means Going Forward — builder2ibek reference, advocacy close + +### Critical Pitfalls + +1. **Inflated or unverifiable productivity claims** — anchor every claim to a verifiable git artifact (commit count, file count, line count, terminal count); "113 commits by one developer in ~2 weeks" is verifiable; "equivalent to N months of traditional development" is not unless the reasoning is shown explicitly +2. **Framing AI as the agent, developer as passive** — the developer (Giles Knap) made all architectural decisions and validated all outputs; Claude executed at scale under direction; review every sentence for subject and agency; the developer should be the grammatical subject of all decision sentences +3. **Scale metrics without quality evidence** — pair each scale metric with a quality indicator (e.g., "113 commits" + "test suite includes full ADS simulator"; "29 terminal types in YAML" + "validated against Beckhoff ESI XML"); the before/after architecture comparison is itself quality evidence +4. **Failing to acknowledge limitations and conditions** — state what predates structured workflows, what AI was not used for, and that the developer's pre-existing domain expertise was essential; this framing increases credibility rather than undermining it +5. **Over-generalising from a single case** — use evidential language throughout: "this project demonstrates..." not "AI-assisted development enables..."; the trajectory arc (builder2ibek) partially addresses this by showing the approach is reproducible + +## Implications for Roadmap + +Based on research, the document has a clear sequential writing dependency chain. Three phases are suggested. + +### Phase 1: Evidence Verification and Metrics Baseline + +**Rationale:** All writing depends on having verified, agreed numbers. Write the metrics table first and confirm the figures with the author (Giles Knap) before any prose is drafted. This avoids rework if numbers are disputed or scoped differently (e.g., architecture-only vs. all commits including tests and docs). + +**Delivers:** A verified, agreed metrics table that all subsequent prose references. Author sign-off on which numbers to use and how to scope them (e.g., 113 commits vs. 114; ~23k new source lines vs. 35,406 total insertions). Author input on the scale-to-team-equivalent estimate (if included) and the builder2ibek trajectory context. + +**Addresses:** "Concrete numbers" (P1 feature); prevents the Pitfall 1 failure mode (unverifiable claims). Author must also provide: acknowledgement of AI limitations, builder2ibek description for the trajectory close. + +**Avoids:** The "writing summary without reading commit history" technical debt pattern identified in PITFALLS.md. + +### Phase 2: Core Document Drafting + +**Rationale:** With metrics verified, draft all six sections in writing-dependency order (metrics table → before-state → after-state → mechanism → introduction → trajectory). The architecture research specifies this order explicitly because introduction cannot be written well until all other sections exist. + +**Delivers:** A complete draft of `docs/explanations/ai-assisted-development.md` with all P1 features present: context paragraph, scale of changes, before/after architectural comparison, where AI was used, honest framing of AI role, advocacy conclusion. + +**Uses:** MyST Markdown in the house style established by existing explanation pages; no new format features needed. The before/after comparison uses a table (Pattern 2 from ARCHITECTURE.md). Metrics use a table block (Pattern 3). Each paragraph in the evidence sections uses the Claim-Evidence-Implication pattern (Pattern 1). + +**Implements:** All six document sections. Applies the five anti-pattern avoidance rules: lead with achievement not process; show impact not task list; metrics in a table not buried in prose; architecture diagram not code; clear advocacy conclusion. + +**Avoids:** Pitfalls 2 (AI as hero), 3 (no limitations), 4 (quantity without quality). + +### Phase 3: Validation, Revision, and Integration + +**Rationale:** After drafting, two categories of review are needed: language audit (check universal claims, check agency, check limitation acknowledgements) and technical validation (Sphinx build, format compliance, cross-reference correctness). These must happen after drafting, not during, to avoid premature optimisation. + +**Delivers:** A document that renders cleanly in the existing Sphinx build, passes `tox -e docs` with `--fail-on-warning`, is compliant with MyST house style, and has been reviewed for evidential language (vs. universal claims) and developer agency (vs. passive AI-hero framing). P2 features (trajectory arc, limitations acknowledgement) finalized. builder2ibek reference verified as accessible. + +**Avoids:** Pitfall 5 (over-generalisation); Sphinx/MyST format integration gotchas (GitHub-flavoured Markdown features, non-supported cross-reference syntax, unverified Mermaid rendering). + +### Phase Ordering Rationale + +- Phase 1 must precede Phase 2: prose cannot be written accurately without agreed numbers; author input cannot be retrofitted without rework +- Phase 2 must precede Phase 3: language audit requires the full draft to exist; Sphinx build validation requires a draft to render +- The writing-dependency order within Phase 2 (metrics table first, introduction last) is specified by ARCHITECTURE.md and reflects rhetorical dependencies within the document itself + +### Research Flags + +Phases needing deeper research or author input during planning: +- **Phase 1:** Author (Giles Knap) must confirm: (a) which commit and line count figures to use, (b) whether the scale-to-team-equivalent estimate is included and what the number is, (c) what the builder2ibek trajectory description should say, (d) which limitations to acknowledge +- **Phase 2:** The "How AI Assistance Enabled This" section requires the author's retrospective account of specific tasks — this cannot be accurately drafted from the commit history alone without risking misattribution + +Phases with standard patterns where research is not needed: +- **Phase 3:** Sphinx build validation, MyST format compliance, and language audit are standard review steps with clear checklists already defined in PITFALLS.md; no additional research required + +## Confidence Assessment + +| Area | Confidence | Notes | +|------|------------|-------| +| Stack | HIGH | All technologies verified directly in `pyproject.toml`, `conf.py`, and existing explanation pages; no inference required | +| Features | MEDIUM | MVP content list is well-grounded in project evidence and established advocacy document patterns; web search unavailable to verify against current external practice | +| Architecture | HIGH | Document section structure and rhetorical patterns are derived from first principles of persuasion architecture; verified against existing explanation pages for format; writing-order dependency chain is directly extractable from the content relationships | +| Pitfalls | MEDIUM | All five critical pitfalls are grounded in project-specific evidence and established technical writing principles; web search unavailable to verify against recent AI advocacy document failures | + +**Overall confidence:** MEDIUM-HIGH. The infrastructure and evidence base are fully verified (HIGH). The content and rhetorical recommendations are well-grounded but draw on training knowledge rather than current external sources (MEDIUM). No decisions here require LOW-confidence information. + +### Gaps to Address + +- **Author input required before drafting:** The metrics scoping decision (architecture-only vs. all work), the scale-to-team-equivalent estimate (if included), the builder2ibek trajectory description, and the limitations acknowledgement all require input from Giles Knap and cannot be derived from the repository alone. +- **Mermaid diagram decision:** The before/after architecture diagram is deferred to v2+ but should be revisited after the text comparison table is drafted; if stakeholders or reviewers find the text comparison insufficient, the Mermaid diagram moves to Phase 2. +- **builder2ibek accessibility:** The builder2ibek reference must be verified as accessible to the target audience (DLS management/stakeholders) before the document is published; if it is not publicly accessible, the trajectory section must be rewritten to describe the project without linking to it. + +## Sources + +### Primary (HIGH confidence) +- `/scratch/hgv27681/work/fastcs-catio/.planning/PROJECT.md` — project requirements, audience definition, known metrics, constraints +- `/scratch/hgv27681/work/fastcs-catio/docs/conf.py` — Sphinx configuration, verified extensions +- `/scratch/hgv27681/work/fastcs-catio/docs/explanations/` — existing explanation page house format (architecture-overview.md, ads-client.md, fastcs-epics-ioc.md, terminal-yaml-definitions.md) +- `/scratch/hgv27681/work/fastcs-catio/CLAUDE_NOTES.md` — first-person account of AI-assisted workflow +- `/scratch/hgv27681/work/fastcs-catio/SKILLS.md` — MyST/Mermaid patterns validated in project +- Git history (`git log`, `git diff df5c3c4..HEAD --stat`) — commit counts, file counts, timeline (verified) + +### Secondary (MEDIUM confidence) +- Established persuasion architecture principles (training knowledge) — document section order and rhetorical patterns +- Technical writing practice for stakeholder advocacy documents (training knowledge) — what management audiences require to find claims credible + +### Tertiary (LOW confidence) +- None: no low-confidence sources used; where training knowledge was applied it is flagged as MEDIUM + +--- +*Research completed: 2026-03-02* +*Ready for roadmap: yes* From d044861fc8febf704261d4178a81c48f0541183c Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:37:53 +0000 Subject: [PATCH 06/12] docs: define v1 requirements --- .planning/REQUIREMENTS.md | 68 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 .planning/REQUIREMENTS.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md new file mode 100644 index 0000000..fb27091 --- /dev/null +++ b/.planning/REQUIREMENTS.md @@ -0,0 +1,68 @@ +# Requirements: AI Agents Executive Summary + +**Defined:** 2026-03-02 +**Core Value:** Demonstrate through concrete evidence that AI-assisted development delivered a major architectural overhaul rapidly and with high quality + +## v1 Requirements + +### Content + +- [ ] **CONT-01**: Document includes a concise before-state description (hard-coded Python classes per terminal type) +- [ ] **CONT-02**: Document includes quantified scale metrics from git history (commits, lines, files, terminal types) +- [ ] **CONT-03**: Document includes high-level overview of the architectural transformation (YAML-driven dynamic controllers, catio-terminals GUI) +- [ ] **CONT-04**: Document identifies where Claude Opus was used to achieve rapid development +- [ ] **CONT-05**: Document states the timeline (~2 weeks, single developer) +- [ ] **CONT-06**: Document frames the developer as the agent who directed AI, not AI as autonomous hero + +### Advocacy + +- [ ] **ADVC-01**: Document includes trajectory arc — this work predates agent skills, builder2ibek shows maturation +- [ ] **ADVC-02**: Document highlights maintainability improvement (new terminal = edit YAML, not write Python) + +### Format + +- [ ] **FRMT-01**: Document is a Sphinx/MyST markdown page in docs/explanations/ +- [ ] **FRMT-02**: Document is registered in docs/explanations.md toctree +- [ ] **FRMT-03**: Document is short and scannable — executive summary, not a report + +## v2 Requirements + +### Content + +- **CONT-07**: Before/after architecture diagram (mermaid) +- **CONT-08**: Quality evidence (test coverage, validation against Beckhoff XML) + +## Out of Scope + +| Feature | Reason | +|---------|--------| +| Step-by-step AI tutorial | Wrong audience — stakeholders, not developers | +| Detailed code examples | Executive summary, not technical reference | +| Productivity multiplier claims | Undermines credibility — let raw numbers speak | +| Test/doc/cleanup commit coverage | Architecture focus only per user direction | +| Lengthy prose sections | Must be short and snappy | + +## Traceability + +| Requirement | Phase | Status | +|-------------|-------|--------| +| CONT-01 | Phase 1 | Pending | +| CONT-02 | Phase 1 | Pending | +| CONT-03 | Phase 1 | Pending | +| CONT-04 | Phase 1 | Pending | +| CONT-05 | Phase 1 | Pending | +| CONT-06 | Phase 1 | Pending | +| ADVC-01 | Phase 1 | Pending | +| ADVC-02 | Phase 1 | Pending | +| FRMT-01 | Phase 1 | Pending | +| FRMT-02 | Phase 1 | Pending | +| FRMT-03 | Phase 1 | Pending | + +**Coverage:** +- v1 requirements: 11 total +- Mapped to phases: 11 +- Unmapped: 0 ✓ + +--- +*Requirements defined: 2026-03-02* +*Last updated: 2026-03-02 after initial definition* From 5fbd486a5de091346fbd2bd31d19156ed6e08142 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:40:12 +0000 Subject: [PATCH 07/12] docs: create roadmap (2 phases) --- .planning/REQUIREMENTS.md | 10 +++---- .planning/ROADMAP.md | 56 +++++++++++++++++++++++++++++++++++++ .planning/STATE.md | 59 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 120 insertions(+), 5 deletions(-) create mode 100644 .planning/ROADMAP.md create mode 100644 .planning/STATE.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index fb27091..ef62511 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -54,15 +54,15 @@ | CONT-06 | Phase 1 | Pending | | ADVC-01 | Phase 1 | Pending | | ADVC-02 | Phase 1 | Pending | -| FRMT-01 | Phase 1 | Pending | -| FRMT-02 | Phase 1 | Pending | -| FRMT-03 | Phase 1 | Pending | +| FRMT-01 | Phase 2 | Pending | +| FRMT-02 | Phase 2 | Pending | +| FRMT-03 | Phase 2 | Pending | **Coverage:** - v1 requirements: 11 total - Mapped to phases: 11 -- Unmapped: 0 ✓ +- Unmapped: 0 --- *Requirements defined: 2026-03-02* -*Last updated: 2026-03-02 after initial definition* +*Last updated: 2026-03-02 after roadmap creation* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md new file mode 100644 index 0000000..9a0dba9 --- /dev/null +++ b/.planning/ROADMAP.md @@ -0,0 +1,56 @@ +# Roadmap: AI Agents Executive Summary + +## Overview + +Deliver a single Sphinx explanation page that makes the case for AI-assisted development using the CATio architectural transformation as evidence. Phase 1 writes the document content (evidence, narrative, advocacy). Phase 2 integrates it into the Sphinx docs build and validates format compliance. + +## Phases + +**Phase Numbering:** +- Integer phases (1, 2, 3): Planned milestone work +- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) + +Decimal phases appear between their surrounding integers in numeric order. + +- [ ] **Phase 1: Document Content** - Draft the executive summary with all evidence, narrative, and advocacy sections +- [ ] **Phase 2: Sphinx Integration** - Place the page in the docs tree, register in toctree, validate build + +## Phase Details + +### Phase 1: Document Content +**Goal**: A complete draft of the executive summary exists with verified metrics, before/after narrative, honest AI framing, and advocacy arc +**Depends on**: Nothing (first phase) +**Requirements**: CONT-01, CONT-02, CONT-03, CONT-04, CONT-05, CONT-06, ADVC-01, ADVC-02 +**Success Criteria** (what must be TRUE): + 1. Document contains a metrics table with verifiable numbers from git history (commits, lines, files, terminal types) + 2. Document describes the before-state (hard-coded Python) and after-state (YAML-driven dynamic controllers) as a clear transformation arc + 3. Document identifies specific tasks where Claude Opus was used and frames the developer as the directing agent + 4. Document includes the trajectory arc from pre-agent CATio work to builder2ibek as evidence of maturing practice + 5. Document reads as a concise, scannable executive summary -- not a technical report +**Plans**: TBD + +Plans: +- [ ] 01-01: TBD + +### Phase 2: Sphinx Integration +**Goal**: The document is a first-class page in the CATio docs that builds cleanly and is discoverable in the explanations section +**Depends on**: Phase 1 +**Requirements**: FRMT-01, FRMT-02, FRMT-03 +**Success Criteria** (what must be TRUE): + 1. Document exists at docs/explanations/ai-assisted-development.md in valid MyST markdown + 2. Document appears in the explanations toctree and is navigable from the docs site + 3. `tox -e docs` passes with no warnings related to this page +**Plans**: TBD + +Plans: +- [ ] 02-01: TBD + +## Progress + +**Execution Order:** +Phases execute in numeric order: 1 -> 2 + +| Phase | Plans Complete | Status | Completed | +|-------|----------------|--------|-----------| +| 1. Document Content | 0/? | Not started | - | +| 2. Sphinx Integration | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md new file mode 100644 index 0000000..5e1b050 --- /dev/null +++ b/.planning/STATE.md @@ -0,0 +1,59 @@ +# Project State + +## Project Reference + +See: .planning/PROJECT.md (updated 2026-03-02) + +**Core value:** Demonstrate through concrete evidence that AI-assisted development delivered a major architectural overhaul rapidly and with high quality +**Current focus:** Phase 1: Document Content + +## Current Position + +Phase: 1 of 2 (Document Content) +Plan: 0 of ? in current phase +Status: Ready to plan +Last activity: 2026-03-02 -- Roadmap created + +Progress: [..........] 0% + +## Performance Metrics + +**Velocity:** +- Total plans completed: 0 +- Average duration: - +- Total execution time: 0 hours + +**By Phase:** + +| Phase | Plans | Total | Avg/Plan | +|-------|-------|-------|----------| +| - | - | - | - | + +**Recent Trend:** +- Last 5 plans: - +- Trend: - + +*Updated after each plan completion* + +## Accumulated Context + +### Decisions + +Decisions are logged in PROJECT.md Key Decisions table. +Recent decisions affecting current work: + +- [Roadmap]: Two phases -- content drafting then Sphinx integration. Research suggested three phases but the deliverable is a single page; two natural boundaries suffice. + +### Pending Todos + +None yet. + +### Blockers/Concerns + +- Author (Giles Knap) input may be needed to confirm exact metric scoping (architecture-only vs all commits) and builder2ibek trajectory description. Research flagged this; plan-phase should account for it. + +## Session Continuity + +Last session: 2026-03-02 +Stopped at: Roadmap created, ready for Phase 1 planning +Resume file: None From c708497d7cd5a1089e559bbe07e5a5e13ffadb06 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 11:45:39 +0000 Subject: [PATCH 08/12] docs(01): capture phase context --- .../phases/01-document-content/01-CONTEXT.md | 88 +++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 .planning/phases/01-document-content/01-CONTEXT.md diff --git a/.planning/phases/01-document-content/01-CONTEXT.md b/.planning/phases/01-document-content/01-CONTEXT.md new file mode 100644 index 0000000..981a479 --- /dev/null +++ b/.planning/phases/01-document-content/01-CONTEXT.md @@ -0,0 +1,88 @@ +# Phase 1: Document Content - Context + +**Gathered:** 2026-03-02 +**Status:** Ready for planning + + +## Phase Boundary + +Draft a complete executive summary documenting the CATio architectural transformation as evidence for AI-assisted development. Includes verified metrics, before/after narrative, honest AI framing, and advocacy arc. Sphinx integration and formatting are Phase 2. + + + + +## Implementation Decisions + +### Narrative structure +- Outcome-first opening — lead with the transformation result (YAML-driven dynamic controllers, ~2 weeks, one developer + AI) then explain how +- Two-paragraph before/after contrast — enough to understand the pain (hard-coded Python per terminal type) and appreciate the solution, without repeating existing technical docs +- Trajectory arc (builder2ibek → CATio) woven into the main narrative, not a separate section +- Forward-looking close — end on maintainability payoff ("new terminal = edit YAML, not write Python") as proof the investment pays forward + +### Evidence presentation +- Both inline numbers in prose AND a summary metrics table for quick reference +- Metrics cover the full scope: architecture transformation AND catio-terminals GUI tool +- One minimal before/after code snippet (3-5 lines each) to make the transformation tangible +- Link to existing detailed docs for readers who want to go deeper + +### AI framing tone +- Developer-AI collaboration framing — "pair programming" where developer provided architectural vision, Claude handled bulk implementation +- Task-level specifics on what Claude did — name concrete tasks (controller boilerplate, protocol handlers, etc.) +- Brief honest mention of limitations/challenges to build credibility +- Brief paragraph (2-3 sentences) on builder2ibek as evidence of maturing AI-assisted practice +- Developer positioned as the directing agent throughout (CONT-06) + +### Audience & intent +- Mixed audience: technical managers, fellow developers, and semi-technical stakeholders +- Dual takeaway: appreciation for the achievement + curiosity about AI adoption +- Domain-specific terms (EtherCAT, CoE, PDO, ADS) used with brief inline context on first mention +- Cross-references to existing explanation pages (architecture overview, terminal YAML definitions, etc.) + +### Claude's Discretion +- Metric precision (exact vs rounded numbers) — whatever reads naturally and stays credible +- Loading skeleton design for any diagrams or visual elements +- Exact section headings and document length within "executive summary" constraint + + + + +## Specific Ideas + +- "Let raw numbers speak" — no productivity multiplier claims, present the data and let readers draw conclusions +- builder2ibek should be positioned as the earlier, simpler AI project that paved the way +- The requirements explicitly exclude: step-by-step AI tutorials, detailed code examples, and lengthy prose sections +- Document should be "short and snappy" per FRMT-03 + + + + +## Existing Code Insights + +### Reusable Assets +- `docs/explanations/architecture-overview.md`: Existing architecture documentation with mermaid diagrams — can cross-reference rather than duplicate +- `docs/explanations/terminal-yaml-definitions.md`: Existing YAML config documentation — reference for the "after-state" +- `docs/explanations/fastcs-epics-ioc.md`: Documents dynamic controller generation — reference material + +### Established Patterns +- MyST markdown format for all docs in `docs/explanations/` +- Toctree registration in `docs/explanations.md` +- Mermaid diagrams with `mermaid-simple` CSS class for font sizing + +### Integration Points +- New page will be added to `docs/explanations/` directory (Phase 2 handles toctree registration) +- Git history provides verifiable metrics (major refactoring: 72 files, 9111 insertions in one commit) +- Two packages in scope: `fastcs_catio` (controller framework) and `catio_terminals` (GUI tool) + + + + +## Deferred Ideas + +None — discussion stayed within phase scope + + + +--- + +*Phase: 01-document-content* +*Context gathered: 2026-03-02* From 6ec34728947d4e36942c5271ea573264e6fd9b54 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 12:24:09 +0000 Subject: [PATCH 09/12] docs(01): research phase domain --- .../phases/01-document-content/01-RESEARCH.md | 417 ++++++++++++++++++ 1 file changed, 417 insertions(+) create mode 100644 .planning/phases/01-document-content/01-RESEARCH.md diff --git a/.planning/phases/01-document-content/01-RESEARCH.md b/.planning/phases/01-document-content/01-RESEARCH.md new file mode 100644 index 0000000..5488941 --- /dev/null +++ b/.planning/phases/01-document-content/01-RESEARCH.md @@ -0,0 +1,417 @@ +# Phase 1: Document Content - Research + +**Researched:** 2026-03-02 +**Domain:** Technical writing -- executive summary of AI-assisted architectural transformation +**Confidence:** HIGH + +## Summary + +This phase produces the content of an executive summary documenting the CATio architectural transformation. The document is a narrative piece, not a software engineering deliverable -- the "stack" is document structure, evidence presentation, and writing discipline. All the raw evidence already exists in the git history and codebase: 16 source files grew to 56, 20 hard-coded Python terminal controller classes were replaced by 29 YAML-defined terminal types with dynamic controller generation, and a separate `catio_terminals` GUI tool was created from scratch -- all within roughly two weeks by a single developer using Claude Opus. + +The critical research finding is that the before/after contrast is concretely demonstrable through the codebase. The old `catio_hardware.py` contained 1199 lines of hand-written Python with 20 explicit controller classes (one per terminal type), each manually defining attributes via `add_attribute()` calls. The new approach uses a single factory function (`get_terminal_controller_class()`) that reads YAML definitions and generates controller classes at runtime. Adding a new terminal type went from "write a new Python class with 30-60 lines of boilerplate" to "edit a YAML file." This is the core narrative. + +**Primary recommendation:** Structure the document as outcome-first prose with an embedded metrics table, a minimal before/after code snippet (old hard-coded class vs YAML definition), and a brief trajectory paragraph referencing builder2ibek. Keep it under 300 lines of MyST markdown. Every factual claim must trace to a git commit, file count, or line count. + + +## User Constraints (from CONTEXT.md) + +### Locked Decisions +- **Narrative structure**: Outcome-first opening -- lead with the transformation result (YAML-driven dynamic controllers, ~2 weeks, one developer + AI) then explain how +- **Two-paragraph before/after contrast** -- enough to understand the pain (hard-coded Python per terminal type) and appreciate the solution, without repeating existing technical docs +- **Trajectory arc (builder2ibek -> CATio)** woven into the main narrative, not a separate section +- **Forward-looking close** -- end on maintainability payoff ("new terminal = edit YAML, not write Python") as proof the investment pays forward +- **Both inline numbers in prose AND a summary metrics table** for quick reference +- **Metrics cover the full scope**: architecture transformation AND catio-terminals GUI tool +- **One minimal before/after code snippet** (3-5 lines each) to make the transformation tangible +- **Link to existing detailed docs** for readers who want to go deeper +- **Developer-AI collaboration framing** -- "pair programming" where developer provided architectural vision, Claude handled bulk implementation +- **Task-level specifics on what Claude did** -- name concrete tasks (controller boilerplate, protocol handlers, etc.) +- **Brief honest mention of limitations/challenges** to build credibility +- **Brief paragraph (2-3 sentences) on builder2ibek** as evidence of maturing AI-assisted practice +- **Developer positioned as the directing agent throughout** (CONT-06) +- **Mixed audience**: technical managers, fellow developers, and semi-technical stakeholders +- **Dual takeaway**: appreciation for the achievement + curiosity about AI adoption +- **Domain-specific terms** (EtherCAT, CoE, PDO, ADS) used with brief inline context on first mention +- **Cross-references to existing explanation pages** (architecture overview, terminal YAML definitions, etc.) +- **"Let raw numbers speak"** -- no productivity multiplier claims, present the data and let readers draw conclusions +- **builder2ibek positioned as the earlier, simpler AI project** that paved the way +- **Explicitly exclude**: step-by-step AI tutorials, detailed code examples, lengthy prose sections +- **"Short and snappy"** per FRMT-03 + +### Claude's Discretion +- Metric precision (exact vs rounded numbers) -- whatever reads naturally and stays credible +- Loading skeleton design for any diagrams or visual elements +- Exact section headings and document length within "executive summary" constraint + +### Deferred Ideas (OUT OF SCOPE) +None -- discussion stayed within phase scope + + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|-----------------| +| CONT-01 | Document includes a concise before-state description (hard-coded Python classes per terminal type) | Git evidence at commit `e8e2f37`: `catio_hardware.py` was 1199 lines with 20 explicit controller classes. Each terminal type required a dedicated Python class with manual `add_attribute()` calls (see Code Examples section). | +| CONT-02 | Document includes quantified scale metrics from git history (commits, lines, files, terminal types) | Full metrics table verified from git: ~141 non-merge commits by Giles (Dec 15 - Feb 5), 16->56 source files, 29 terminal types in YAML, 14,276 lines of YAML, ~17,680 lines of Python across both packages. See Verified Metrics section. | +| CONT-03 | Document includes high-level overview of architectural transformation (YAML-driven dynamic controllers, catio-terminals GUI) | Architecture documented in existing `architecture-overview.md` and `terminal-yaml-definitions.md`. Dynamic controller factory in `catio_dynamic_controller.py`. GUI tool is `catio_terminals` package with NiceGUI web editor. See Architecture Patterns section. | +| CONT-04 | Document identifies where Claude Opus was used to achieve rapid development | `CLAUDE_NOTES.md` documents specific AI usage: architecture docs generation (commit `4dcc9d5`), code refactoring when complexity grew (Opus for analysis, Sonnet for implementation), feature documentation before coding, skill extraction to AGENTS.md. | +| CONT-05 | Document states the timeline (~2 weeks, single developer) | Git history confirms: first major commit Dec 15 2025 (`399ff48`), main transformation Jan 22 - Feb 5 2026. Single developer (Giles Knap) authored ~141 of ~154 non-merge commits. | +| CONT-06 | Document frames the developer as the agent who directed AI, not AI as autonomous hero | `CLAUDE_NOTES.md` provides first-person evidence: developer directed all architecture, used Opus for analysis/refactoring, Sonnet for implementation. PITFALLS.md documents the anti-pattern and prevention strategy. | +| ADVC-01 | Document includes trajectory arc -- this work predates agent skills, builder2ibek shows maturation | `CLAUDE_NOTES.md` confirms pre-agent workflow (chat window, manual skill extraction). `builder2ibek` exists as sibling project at `/scratch/hgv27681/work/builder2ibek/` with structured agent features. | +| ADVC-02 | Document highlights maintainability improvement (new terminal = edit YAML, not write Python) | Before: adding a terminal required writing 30-60 lines of Python in `catio_hardware.py`. After: adding a terminal means adding ~15-25 lines of YAML to `terminal_types.yaml` (or using the GUI editor). 10 new terminal types were added after the initial set with zero Python changes. | + + +## Standard Stack + +### Core + +This phase produces a document, not software. The "stack" is the writing approach and evidence sources. + +| Component | Location | Purpose | Why Standard | +|-----------|----------|---------|--------------| +| MyST Markdown | `docs/explanations/` | Document format | All existing explanation pages use this format; zero setup needed | +| Git history | Repository | Primary evidence source | All metrics claims must be verifiable from git | +| `CLAUDE_NOTES.md` | Repository root | AI workflow evidence | First-person developer account of Claude usage | +| `AGENTS.md` | Repository root | AI context documentation | Shows how project-specific AI context was managed | +| Existing explanation pages | `docs/explanations/` | Cross-reference targets | Architecture overview, terminal YAML definitions, FastCS IOC docs | + +### Supporting + +| Component | Location | Purpose | When to Use | +|-----------|----------|---------|-------------| +| sphinx-design admonitions | In-page `:::` syntax | Callout boxes for caveats | One `note` for "predates workflows" caveat; sparingly | +| Mermaid diagrams | In-page code blocks | Visual architecture contrast | Only if a simple before/after diagram adds value; not required by user decisions | +| Relative links | MyST `[text](path.md)` | Cross-references to sibling docs | Link to architecture-overview.md, terminal-yaml-definitions.md, etc. | + +### Alternatives Considered + +| Recommended | Alternative | Tradeoff | +|-------------|-------------|----------| +| Single markdown file | Multi-file with includes | Complexity for no benefit -- this is one page | +| Inline metrics table | Separate data file | Table is small enough (8-10 rows) to inline | +| Minimal code snippet (3-5 lines) | Full before/after module comparison | Executive audience -- one snippet is enough to make the point tangible | + +## Architecture Patterns + +### Recommended Document Structure + +``` +# AI-Assisted Architectural Transformation of CATio + +[Opening paragraph: outcome-first -- what was delivered, scale, timeline] + +## The Transformation +[Before-state paragraph: hard-coded Python, maintenance burden] +[After-state paragraph: YAML-driven, dynamic generation, GUI tool] +[Minimal code snippet: before vs after] + +## By the Numbers +[Metrics table: commits, files, lines, terminal types, timeline] + +## How AI Accelerated Development +[What Claude did: concrete tasks named] +[Developer-AI collaboration framing] +[Honest limitations mention] +[builder2ibek trajectory woven in] + +## What This Means Going Forward +[Maintainability payoff: new terminal = edit YAML] +[Forward-looking close] + +## Learn More +[Cross-references to existing docs] +``` + +### Pattern: Outcome-First Opening + +**What:** Lead with the transformation result, not the problem. +**When to use:** Always for executive summaries. Busy readers need the conclusion before the reasoning. +**Example:** +``` +A single developer, working with Claude Opus over approximately two weeks, +transformed CATio from a system requiring hand-written Python for each +terminal type to one driven entirely by YAML definitions -- covering 29 +terminal types across two packages and roughly 17,000 lines of Python. +``` + +### Pattern: Paired Evidence (Scale + Quality) + +**What:** Every scale metric is immediately followed by a quality indicator. +**When to use:** For every quantitative claim. +**Example:** +``` +| Metric | Value | Quality Indicator | +|--------|-------|-------------------| +| Terminal types | 29 in YAML | Validated against Beckhoff ESI XML | +| Source files | 16 -> 56 | Documented in 10 explanation pages | +``` + +### Anti-Patterns to Avoid + +- **Productivity multiplier claims:** Never write "10x faster" or "saved 6 months." Present raw numbers and let readers draw conclusions. +- **AI as hero:** Never make Claude the grammatical subject of architectural decisions. "The developer directed; Claude executed." +- **Passive voice for AI contributions:** "The code was generated" obscures agency. Write "Claude generated the controller boilerplate under the developer's direction." +- **Jargon without context:** First mention of EtherCAT, CoE, PDO, ADS must include a brief inline explanation (e.g., "CoE (CANopen over EtherCAT) configuration parameters"). + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| Metrics computation | Don't manually count commits or lines | Git commands verified against the repository | Accuracy -- manual counts drift from reality | +| Before/after code snippet | Don't write synthetic "example" code | Extract from actual git history (`git show e8e2f37:src/...`) | Authenticity -- real code is more credible than illustrations | +| builder2ibek description | Don't guess at what builder2ibek does | Check the sibling project's README.md | Accuracy -- the reference must match reality | + +**Key insight:** This is a document about evidence. Every claim must trace to a verifiable artifact. Don't synthesise or approximate when the real data is available. + +## Common Pitfalls + +### Pitfall 1: Unverifiable Productivity Claims + +**What goes wrong:** The document says "10x faster" without grounding in specifics. Skeptical stakeholders dismiss the entire document. +**Why it happens:** Authors reach for dramatic framing to convey impact. +**How to avoid:** Anchor every claim in verifiable facts. "141 commits over two weeks by one developer" is verifiable. "Equivalent to 6 months of traditional development" is not. Present raw numbers; let readers draw conclusions. +**Warning signs:** Any sentence with "X times faster" that lacks a specific baseline measurement. + +### Pitfall 2: AI Treated as Autonomous Agent + +**What goes wrong:** The narrative frames Claude as the agent that delivered the work, sidelining the developer's judgment, direction, and domain expertise. +**Why it happens:** AI tools are novel and attract disproportionate attention in the narrative. +**How to avoid:** Developer (Giles Knap) is the grammatical subject of all decisions. Claude is the subject of execution tasks. "The developer designed the YAML schema; Claude generated the 29 terminal definitions from Beckhoff XML." +**Warning signs:** Passive constructions like "the code was generated" or "the architecture emerged." + +### Pitfall 3: Scale Without Quality Evidence + +**What goes wrong:** The document leads with "17,000 lines of new code" but never demonstrates the code is correct or maintainable. This backfires with technical audiences who know large quantities of AI-generated code can be low quality. +**Why it happens:** Scale is easy to measure; quality is harder to articulate. +**How to avoid:** Pair each scale metric with a quality indicator: terminal definitions validated against Beckhoff ESI XML, system test suite, architecture documented in explanation pages, code refactored into smaller modules when complexity grew. +**Warning signs:** Scale metrics in the introduction with no quality follow-through anywhere. + +### Pitfall 4: Omitting Conditions for Success + +**What goes wrong:** The document implies AI-assisted development works universally without noting that this succeeded because of a developer with deep domain expertise directing the tool. +**Why it happens:** Advocacy tone tempts authors to omit caveats. +**How to avoid:** Frame as evidence from one project, not proof of a universal method. Mention the developer's pre-existing EtherCAT/TwinCAT/ADS expertise. The builder2ibek trajectory reference naturally addresses this by showing the approach has matured. +**Warning signs:** Language like "AI-assisted development enables..." (universal) rather than "this project demonstrates..." (evidential). + +### Pitfall 5: Inaccurate Metrics + +**What goes wrong:** Numbers quoted in the document don't match what git history shows. A single factual error undermines the entire evidence base. +**Why it happens:** Metrics are computed once during research and not re-verified against the source. +**How to avoid:** Every number in the document must be reproducible from a specific git command. Include the verification commands in plan task descriptions so the implementer can re-run them. +**Warning signs:** Round numbers that look estimated (e.g., "about 150 commits" when the actual count is 141). + +## Code Examples + +### Before State: Hard-Coded Terminal Controller (from git history) + +Source: `git show e8e2f37:src/fastcs_catio/catio_hardware.py` (lines 215-270) + +```python +class EL1004Controller(CATioTerminalController): + """A sub-controller for an EL1004 EtherCAT digital input terminal.""" + io_function: str = "4-channel digital input, 24V DC, 3ms filter" + num_channels: int = 4 + + async def get_io_attributes(self) -> None: + initial_attr_count = len(self.attributes) + await super().get_io_attributes() + self.add_attribute("WcState", AttrR(datatype=Int(), ...)) + self.add_attribute("InputToggle", AttrR(datatype=Int(), ...)) + for i in range(1, self.num_channels + 1): + self.add_attribute(f"DICh{i}Value", AttrR(datatype=Int(), ...)) + self.ads_name_map[f"DICh{i}Value"] = f"Channel{i}" +``` + +Each terminal type required a dedicated Python class. 20 classes totalling 1199 lines. Adding a new terminal type meant writing 30-60 lines of boilerplate. + +### After State: YAML Terminal Definition (current) + +Source: `src/catio_terminals/terminals/terminal_types.yaml` + +```yaml +EL1004: + description: 4Ch. Dig. Input 24V, 3ms + identity: + vendor_id: 2 + product_code: 65810514 + revision_number: 1048576 + symbol_nodes: + - name_template: Channel {channel} + index_group: 61489 + type_name: InputBits + channels: 4 + access: Read-only + fastcs_name: channel_{channel} + selected: true + coe_objects: [] + group_type: DigIn +``` + +The dynamic controller factory (`get_terminal_controller_class()`) reads this YAML at runtime and generates a FastCS controller class. Adding a new terminal type means adding a YAML block (or using the GUI editor). + +### Recommended Minimal Snippet for the Document + +For the executive summary, use an even more condensed version (3-5 lines each side): + +```python +# Before: one Python class per terminal type (x20 classes, 1199 lines) +class EL1004Controller(CATioTerminalController): + io_function = "4-channel digital input, 24V DC, 3ms filter" + async def get_io_attributes(self): + for i in range(1, 5): + self.add_attribute(f"DICh{i}Value", AttrR(datatype=Int(), ...)) +``` + +```yaml +# After: one YAML block per terminal type (29 types, generated from vendor XML) +EL1004: + description: 4Ch. Dig. Input 24V, 3ms + symbol_nodes: + - name_template: Channel {channel} + channels: 4 + selected: true +``` + +## Verified Metrics + +All numbers below are verified from the git repository. Commands are provided for re-verification. + +### Timeline + +| Metric | Value | Verification Command | +|--------|-------|---------------------| +| First major commit | 2025-12-15 (`399ff48`) | `git show 399ff48 --format="%ad" --date=short` | +| Core transformation period | 2026-01-22 to 2026-02-05 | `git log --after="2026-01-21" --before="2026-02-06" --oneline` | +| Primary developer | Giles Knap | `git log --format="%aN" \| sort \| uniq -c` | +| Duration | ~2 weeks (core), ~7 weeks (full Dec-Feb) | Git date range | + +### Scale Metrics + +| Metric | Before | After | Verification | +|--------|--------|-------|--------------| +| Python source files (src/) | 16 | 50 | `git ls-tree -r --name-only {commit} -- src/ \| grep '\.py$' \| wc -l` | +| Total source files (src/) | 16 | 56 | `git ls-tree -r --name-only {commit} -- src/ \| wc -l` | +| Hard-coded terminal classes | 20 (in catio_hardware.py) | 0 needed (dynamic) | `grep "^class " catio_hardware.py` | +| Terminal types supported | 20 (Python only) | 29 (YAML) | `grep "^ EL\|^ EK\|^ ELM" terminal_types.yaml \| wc -l` | +| catio_hardware.py lines | 1199 | 1204 (still present as fallback) | `wc -l catio_hardware.py` | +| YAML terminal definitions | 0 | 14,276 lines | `wc -l terminal_types.yaml` | +| catio_terminals package | Did not exist | 31 Python files, 6,256 lines | `find src/catio_terminals -name '*.py' \| wc -l` | +| fastcs_catio Python lines | ~5,000 (est.) | 11,424 | `find src/fastcs_catio -name '*.py' -exec wc -l {} + \| tail -1` | +| New dynamic modules | 0 | 6 files (catio_dynamic_*.py, terminal_config.py, logging.py) | `ls src/fastcs_catio/catio_dynamic*.py` | + +### Commit Metrics + +| Metric | Value | Verification | +|--------|-------|--------------| +| Non-merge commits (Dec 15 - Feb 5) | 154 | `git log --no-merges --after=... --before=... \| wc -l` | +| Giles Knap commits (all name variants) | ~141 | `git log --author="Giles\|giles\|gilesknap" --no-merges ...` | +| copilot-swe-agent commits | 11 (documentation updates) | `git log --author="copilot-swe-agent" ...` | +| Gregory Gay commits | ~2-15 (varies by name variant) | `git log --author="Gregory\|ggay" ...` | +| Pull requests merged | 13 | `git log --oneline \| grep "Merge pull request"` | + +### Key Milestones + +| Date | Commit | Milestone | +|------|--------|-----------| +| 2025-12-15 | `399ff48` | Upgrade to latest FastCS framework | +| 2026-01-05 | `c1d9b08` | Add initial testing capability (mock server, unit/system/perf tests) | +| 2026-01-13 | `4dcc9d5` | Generate and refine Architecture Docs with Claude | +| 2026-01-22 | `ce8efa1` | Add system test | +| 2026-01-27 | `d2cb4a8` | Add initial version of terminal description YAML editor (catio_terminals born) | +| 2026-01-29 | `2788c9f` | Create dynamic terminal controller class generator | +| 2026-02-02 | `6291d22` | Switch to using YAML for all terminals | +| 2026-02-04 | `fff1a1b` | Refactor dynamic code into smaller modules | +| 2026-02-05 | `a0cc799` | Update documentation to reflect updated code | + +### Metric Precision Recommendation (Claude's Discretion) + +For the document, use these rounded/natural phrasings: +- "approximately two weeks" (not "14 calendar days" or "10 working days") +- "29 terminal types" (exact, small enough to be precise) +- "over 14,000 lines of YAML" (rounded from 14,276 -- exact is fine too) +- "50 Python source files" (rounded from 50 -- actually exact in this case) +- "roughly 140 commits" or "over 140 commits" (from 141 non-merge by Giles) + +## AI Usage Evidence (for CONT-04) + +From `CLAUDE_NOTES.md` and git history, specific Claude tasks include: + +| Task | Evidence | AI Role | Developer Role | +|------|----------|---------|----------------| +| Architecture documentation | Commit `4dcc9d5`: "Generate and refine Architecture Docs with Claude" | Generated initial doc drafts (1,307 lines across 14 files) | Refined and validated accuracy | +| XML parsing / terminal definitions | CLAUDE_NOTES.md: "each time I asked Claude to look in the cached Beckhoff Terminal XML..." | Parsed vendor XML, extracted terminal definitions | Directed what to extract, validated results | +| Code refactoring | CLAUDE_NOTES.md: "hit 12000 lines and Claude Sonnet was struggling... asked Opus to take a look" | Opus analyzed and proposed refactoring; Sonnet implemented | Decided when and how to refactor | +| Feature implementation workflow | CLAUDE_NOTES.md: "get the agent to write a document for the feature first" | Wrote feature docs, then implemented code | Designed workflow, reviewed docs before implementation | +| Controller boilerplate | 20 hard-coded classes -> dynamic factory | Generated repetitive attribute definitions | Designed the YAML schema and factory pattern | +| AGENTS.md / skills | CLAUDE_NOTES.md: multiple references | Codified domain knowledge into reusable context | Identified what knowledge to capture | + +**AI model versions used:** Claude Sonnet 4.5 (primary implementation), Claude Opus 4.5 (analysis, refactoring, complex features). Via GitHub Copilot subscription ($39/month). + +## builder2ibek Trajectory Context (for ADVC-01) + +The builder2ibek project exists at `/scratch/hgv27681/work/builder2ibek/` as a sibling project. Key points for the narrative: + +- CATio work predates structured agent workflows (GSD, agent skills, AGENTS.md skills) +- builder2ibek represents the next stage: built with more mature AI agent features +- The trajectory shows the developer's practice improving over time, not a one-off experiment +- **Open question:** The exact builder2ibek URL and its public accessibility need to be verified before linking. The narrative should work even without a direct link -- describe the progression rather than depending on the URL. + +## Existing Cross-Reference Targets + +Pages the executive summary should link to with `[text](path.md)` relative paths: + +| Page | Path | What It Covers | +|------|------|----------------| +| Architecture Overview | `architecture-overview.md` | Full architecture with Mermaid diagrams, dynamic controller generation | +| Terminal YAML Definitions | `terminal-yaml-definitions.md` | Complete YAML schema, how to add terminals, GUI editor usage | +| FastCS EPICS IOC | `fastcs-epics-ioc.md` | Dynamic controller generation details, runtime attribute creation | +| ADS Client | `ads-client.md` | Protocol implementation details | + +## Open Questions + +1. **builder2ibek public URL** + - What we know: The project exists locally and is referenced in PROJECT.md as a maturation example + - What's unclear: Whether it has a public GitHub URL accessible to the target audience + - Recommendation: Write the builder2ibek paragraph to work without a clickable link. If the URL is confirmed public, add it as enhancement. The planner should flag this as an author-confirmation step. + +2. **Exact metric scoping (architecture-only vs all commits)** + - What we know: PROJECT.md says "113 commits" but git shows ~141 non-merge commits by Giles in the full period. The 113 figure may have been computed differently (e.g., architecture-only, excluding deploy/doc commits). + - What's unclear: Whether the document should use ~141 (all non-merge by Giles) or a filtered count. + - Recommendation: Use the full non-merge count (~141 by Giles, ~154 total) with a note that this includes testing, documentation, and cleanup commits alongside the architecture work. This is more honest and the larger number is still impressive. Flag for author confirmation. + +3. **catio_hardware.py still exists with hard-coded classes** + - What we know: The file still contains 20 hard-coded classes (1204 lines) alongside the new dynamic factory. The routing logic falls back to hard-coded classes first, then dynamic. + - What's unclear: Whether the old classes are retained intentionally (e.g., for tested terminals) or are technical debt. + - Recommendation: Frame accurately -- "dynamic generation handles all 29 YAML-defined terminal types; legacy explicit classes remain as a validated fallback for the original set." This is honest and does not overstate the transformation. + +## Sources + +### Primary (HIGH confidence) +- Git repository history -- all metrics verified against actual commits +- `src/fastcs_catio/catio_hardware.py` -- before-state evidence (hard-coded classes) +- `src/fastcs_catio/catio_dynamic_controller.py` -- after-state evidence (factory function) +- `src/catio_terminals/terminals/terminal_types.yaml` -- YAML terminal definitions +- `CLAUDE_NOTES.md` -- first-person developer account of AI usage +- `docs/explanations/architecture-overview.md` -- existing architecture documentation +- `docs/explanations/terminal-yaml-definitions.md` -- existing YAML documentation + +### Secondary (MEDIUM confidence) +- `.planning/PROJECT.md` -- project context and previously computed metrics (some numbers differ from current git count; flagged in Open Questions) +- `.planning/research/STACK.md` -- previous stack research (document format recommendations) +- `.planning/research/PITFALLS.md` -- previous pitfall research (writing discipline guidance) + +### Tertiary (LOW confidence) +- builder2ibek trajectory details -- project exists locally but public accessibility unverified + +## Metadata + +**Confidence breakdown:** +- Verified metrics: HIGH -- all numbers confirmed from git history with reproducible commands +- Document structure: HIGH -- user decisions in CONTEXT.md are specific and complete +- AI usage evidence: HIGH -- CLAUDE_NOTES.md and commit messages provide direct evidence +- builder2ibek context: MEDIUM -- project exists but URL/accessibility needs confirmation +- Pitfalls/writing discipline: MEDIUM -- based on established technical writing principles (from prior research) + +**Research date:** 2026-03-02 +**Valid until:** 2026-04-02 (stable -- git history does not change; document format is established) From 4906517637fea952bbc7843198413d89853a499e Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 12:28:14 +0000 Subject: [PATCH 10/12] docs(01): create phase plan --- .planning/ROADMAP.md | 6 +- .../phases/01-document-content/01-01-PLAN.md | 314 ++++++++++++++++++ 2 files changed, 317 insertions(+), 3 deletions(-) create mode 100644 .planning/phases/01-document-content/01-01-PLAN.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 9a0dba9..09a90dd 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -27,10 +27,10 @@ Decimal phases appear between their surrounding integers in numeric order. 3. Document identifies specific tasks where Claude Opus was used and frames the developer as the directing agent 4. Document includes the trajectory arc from pre-agent CATio work to builder2ibek as evidence of maturing practice 5. Document reads as a concise, scannable executive summary -- not a technical report -**Plans**: TBD +**Plans**: 1 plan Plans: -- [ ] 01-01: TBD +- [ ] 01-01-PLAN.md -- Verify metrics, write executive summary, author review ### Phase 2: Sphinx Integration **Goal**: The document is a first-class page in the CATio docs that builds cleanly and is discoverable in the explanations section @@ -52,5 +52,5 @@ Phases execute in numeric order: 1 -> 2 | Phase | Plans Complete | Status | Completed | |-------|----------------|--------|-----------| -| 1. Document Content | 0/? | Not started | - | +| 1. Document Content | 0/1 | Planned | - | | 2. Sphinx Integration | 0/? | Not started | - | diff --git a/.planning/phases/01-document-content/01-01-PLAN.md b/.planning/phases/01-document-content/01-01-PLAN.md new file mode 100644 index 0000000..5736d58 --- /dev/null +++ b/.planning/phases/01-document-content/01-01-PLAN.md @@ -0,0 +1,314 @@ +--- +phase: 01-document-content +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - docs/explanations/ai-assisted-development.md +autonomous: false +requirements: + - CONT-01 + - CONT-02 + - CONT-03 + - CONT-04 + - CONT-05 + - CONT-06 + - ADVC-01 + - ADVC-02 + +must_haves: + truths: + - "Document opens with the transformation outcome (YAML-driven, ~2 weeks, one developer + AI) before explaining how" + - "Document contains a metrics table with verifiable numbers from git history" + - "Document describes the before-state (hard-coded Python) and after-state (YAML-driven dynamic controllers)" + - "Document includes a minimal before/after code snippet (3-5 lines each)" + - "Document identifies specific tasks where Claude was used and frames developer as the directing agent" + - "Document weaves builder2ibek trajectory into the main narrative (not a separate section)" + - "Document closes with maintainability payoff (new terminal = edit YAML, not write Python)" + - "Document cross-references existing explanation pages" + - "Domain-specific terms (EtherCAT, CoE, PDO, ADS) include brief inline context on first mention" + - "No productivity multiplier claims appear anywhere in the document" + artifacts: + - path: "docs/explanations/ai-assisted-development.md" + provides: "Complete executive summary of AI-assisted CATio transformation" + contains: "# AI-Assisted" + key_links: + - from: "docs/explanations/ai-assisted-development.md" + to: "docs/explanations/architecture-overview.md" + via: "MyST relative link" + pattern: "architecture-overview\\.md" + - from: "docs/explanations/ai-assisted-development.md" + to: "docs/explanations/terminal-yaml-definitions.md" + via: "MyST relative link" + pattern: "terminal-yaml-definitions\\.md" +--- + + +Draft the complete AI-assisted development executive summary for CATio with verified metrics, before/after narrative, honest AI framing, and advocacy arc. + +Purpose: This is the sole deliverable of Phase 1 -- a single MyST markdown page that makes the case for AI-assisted development using the CATio architectural transformation as concrete evidence. All content decisions were locked in the discuss-phase; this plan executes them. + +Output: `docs/explanations/ai-assisted-development.md` -- a complete, review-ready draft + + + +@/home/hgv27681/.claude/get-shit-done/workflows/execute-plan.md +@/home/hgv27681/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/01-document-content/01-CONTEXT.md +@.planning/phases/01-document-content/01-RESEARCH.md + + + + + + Task 1: Re-verify metrics and extract code snippets from git + docs/explanations/ai-assisted-development.md + +Re-verify the key metrics from the research document against the actual git repository. Run these verification commands and record exact values: + +**Timeline verification:** +```bash +git show 399ff48 --format="%ad" --date=short --no-patch +git log --no-merges --after="2026-01-21" --before="2026-02-06" --oneline | wc -l +``` + +**Scale metrics verification:** +```bash +# Current source file counts +find src/fastcs_catio -name '*.py' | wc -l +find src/catio_terminals -name '*.py' | wc -l +find src/ -name '*.py' | wc -l +# Terminal type count from YAML +grep -c "^ EL\|^ EK\|^ ELM" src/catio_terminals/terminals/terminal_types.yaml +# YAML line count +wc -l src/catio_terminals/terminals/terminal_types.yaml +# Python line counts +find src/fastcs_catio -name '*.py' -exec cat {} + | wc -l +find src/catio_terminals -name '*.py' -exec cat {} + | wc -l +``` + +**Before-state verification:** +```bash +# Extract old hard-coded class count from current catio_hardware.py (legacy fallback) +grep "^class " src/fastcs_catio/catio_hardware.py | wc -l +wc -l src/fastcs_catio/catio_hardware.py +``` + +**Commit metrics:** +```bash +git log --no-merges --after="2025-12-14" --before="2026-02-06" --oneline | wc -l +git log --no-merges --after="2025-12-14" --before="2026-02-06" --author="Giles\|giles\|gilesknap" --oneline | wc -l +``` + +**Code snippet extraction for document:** +Extract the condensed before/after snippet as specified in the research: + +Before (from current catio_hardware.py -- the legacy code still present): +```python +# Before: one Python class per terminal type (x20 classes, 1199 lines) +class EL1004Controller(CATioTerminalController): + io_function = "4-channel digital input, 24V DC, 3ms filter" + async def get_io_attributes(self): + for i in range(1, 5): + self.add_attribute(f"DICh{i}Value", AttrR(datatype=Int(), ...)) +``` + +After (from terminal_types.yaml): +```yaml +# After: one YAML block per terminal type (29 types, generated from vendor XML) +EL1004: + description: 4Ch. Dig. Input 24V, 3ms + symbol_nodes: + - name_template: Channel {channel} + channels: 4 + selected: true +``` + +Record all verified numbers. If any number differs significantly from the research document (01-RESEARCH.md), use the newly verified value and note the discrepancy. + +Do NOT yet create the document file -- just verify metrics and snippets. This prepares the evidence foundation for Task 2. + + + bash -c 'grep -c "^ EL\|^ EK\|^ ELM" src/catio_terminals/terminals/terminal_types.yaml && grep -c "^class " src/fastcs_catio/catio_hardware.py && echo "Metrics verification commands executed successfully"' + + All key metrics from the research document have been re-verified against the live repository. Exact values are recorded for use in the document. Code snippets extracted from actual source files. + + + + Task 2: Write the complete executive summary document + docs/explanations/ai-assisted-development.md + +Create `docs/explanations/ai-assisted-development.md` using the verified metrics from Task 1 and the structure specified in 01-CONTEXT.md and 01-RESEARCH.md. + +**Document structure (from locked decisions):** + +``` +# AI-Assisted Architectural Transformation of CATio + +[Opening paragraph: outcome-first] + - Lead with: YAML-driven dynamic controllers, ~2 weeks, one developer + AI + - Include scale: 29 terminal types, two packages, roughly 17,000 lines of Python + - This is the hook -- busy readers get the conclusion first + +## The Transformation + +[Before-state paragraph] + - Hard-coded Python classes per terminal type (20 classes, ~1200 lines in one file) + - Pain: adding a new terminal = write 30-60 lines of Python boilerplate + - Mention EtherCAT with brief inline context on first use: "EtherCAT (a real-time industrial fieldbus protocol)" + - Keep to ONE paragraph -- enough to understand the pain, don't duplicate existing docs + +[After-state paragraph] + - YAML-driven dynamic controller generation + - Single factory function reads YAML definitions at runtime + - catio_terminals GUI tool for editing terminal definitions + - Adding a new terminal = edit a YAML file (or use the GUI) + - Keep to ONE paragraph + +[Code snippet: before vs after] + - Use the condensed 3-5 line snippets from Task 1 + - Python code block for before, YAML code block for after + - Brief caption for each + +## By the Numbers + +[Metrics table] + - Use verified numbers from Task 1 + - Format as a MyST markdown table + - Include both scale metrics and quality indicators (per research pattern: "Paired Evidence") + - Suggested columns: Metric | Value | Quality Indicator + - Rows: Terminal types, Source files, Python lines, YAML definitions, Timeline, Commits + - Inline numbers also woven into surrounding prose (both table AND inline per locked decision) + +## How AI Accelerated Development + +[Developer-AI collaboration paragraph] + - "Pair programming" framing: developer provided architectural vision, Claude handled bulk implementation + - Name concrete tasks Claude did: controller boilerplate generation, protocol handler implementation, XML parsing for terminal definitions, architecture documentation, code refactoring when complexity grew + - Developer (Giles Knap) is ALWAYS the grammatical subject of decisions + - Claude is the subject of execution tasks only + - Mention AI model versions: Claude Sonnet (primary implementation), Claude Opus (analysis, refactoring) + +[Honest limitations paragraph -- 2-3 sentences] + - Brief mention of challenges (e.g., context limits required splitting into smaller modules, Sonnet struggled with 12,000-line files so Opus was brought in for analysis) + - This builds credibility + +[builder2ibek trajectory -- woven in, NOT a separate section] + - 2-3 sentences positioning builder2ibek as the earlier, simpler AI project + - CATio work predated structured agent workflows + - Shows the developer's AI practice has matured, not a one-off experiment + - Do NOT link to builder2ibek URL (unverified public accessibility) + +**CRITICAL FRAMING RULES (locked decisions + anti-patterns from research):** + - NEVER use "X times faster" or productivity multiplier claims -- let raw numbers speak + - NEVER make Claude the grammatical subject of architectural decisions + - NEVER use passive voice for AI contributions ("the code was generated") -- write "Claude generated X under the developer's direction" + - Frame as evidence from ONE project, not proof of a universal method + - Use "this project demonstrates..." (evidential) not "AI-assisted development enables..." (universal) + - Mention developer's pre-existing domain expertise (EtherCAT/TwinCAT/ADS) + +## What This Means Going Forward + +[Maintainability payoff paragraph] + - Core message: new terminal = edit YAML, not write Python + - 10 new terminal types were added after the initial set with zero Python changes + - Forward-looking: as new Beckhoff terminals enter the market, adding support is a configuration task + - This is the close -- proof the investment pays forward + +## Learn More + +[Cross-references to existing docs] + - Link to architecture-overview.md: full architecture with diagrams + - Link to terminal-yaml-definitions.md: complete YAML schema and how to add terminals + - Link to fastcs-epics-ioc.md: dynamic controller generation details + - Use MyST relative link syntax: `[text](architecture-overview.md)` + +**Format constraints:** + - MyST markdown (consistent with all docs/explanations/ pages) + - Target length: under 300 lines (executive summary, not report) + - Short, scannable sections -- no section longer than ~40 lines + - Domain-specific terms (EtherCAT, CoE, PDO, ADS) get brief inline context on first mention + - No mermaid diagrams unless they clearly add value (Claude's discretion) + - Use rounded/natural number phrasing where appropriate (per research recommendations) + + + bash -c 'test -f docs/explanations/ai-assisted-development.md && wc -l docs/explanations/ai-assisted-development.md | awk "{if (\$1 > 0 && \$1 < 400) print \"OK: \" \$1 \" lines\"; else print \"FAIL: \" \$1 \" lines\"}"' + + + - File `docs/explanations/ai-assisted-development.md` exists with complete content + - Document opens with outcome-first paragraph (YAML-driven, ~2 weeks, one developer + AI) + - Contains metrics table with verifiable numbers + - Contains before/after code snippet (3-5 lines each) + - Names specific AI tasks (controller boilerplate, protocol handlers, XML parsing, etc.) + - Developer is grammatical subject of all decisions; Claude is subject of execution tasks only + - builder2ibek trajectory woven into AI section (not separate section) + - Closes with maintainability payoff + - Cross-references architecture-overview.md, terminal-yaml-definitions.md, fastcs-epics-ioc.md + - No productivity multiplier claims + - Under 300 lines + + + + + Task 3: Author review of executive summary draft + docs/explanations/ai-assisted-development.md + +Present the completed executive summary draft for author (Giles Knap) review. + +**What was built:** +Complete draft of the AI-assisted development executive summary at `docs/explanations/ai-assisted-development.md`. This covers all 8 Phase 1 requirements: before-state description (CONT-01), quantified metrics (CONT-02), architectural transformation overview (CONT-03), AI usage details (CONT-04), timeline (CONT-05), developer-as-agent framing (CONT-06), builder2ibek trajectory (ADVC-01), and maintainability improvement (ADVC-02). + +**How to verify:** +1. Read `docs/explanations/ai-assisted-development.md` +2. Verify the opening paragraph leads with the outcome (not the problem) +3. Verify metrics match your knowledge of the project +4. Verify the AI framing feels right -- you (Giles) are positioned as the directing agent throughout +5. Verify builder2ibek trajectory paragraph is accurate +6. Verify the tone is "short and snappy" executive summary, not a lengthy report +7. Check that domain terms have appropriate context for the mixed audience (technical managers + developers + semi-technical stakeholders) +8. Confirm cross-references to existing docs are correct + +**Open questions flagged during research that may need your input:** +- Metric scoping: document uses full non-merge commit count (~141 by you). Confirm this is preferred over a filtered architecture-only count. +- catio_hardware.py: document frames legacy classes as "validated fallback for the original set." Confirm this is accurate framing. +- builder2ibek: document describes the progression without a URL link. If you want a public URL added, provide it. + +**Resume signal:** Type "approved" to proceed to Phase 2 (Sphinx integration), or describe changes needed. + + + bash -c 'test -f docs/explanations/ai-assisted-development.md && echo "Draft exists for review"' + + Author has reviewed the executive summary draft and either approved it or provided specific change requests + + + + + +Phase 1 verification checks: +1. `docs/explanations/ai-assisted-development.md` exists and is under 300 lines +2. Document contains a metrics table (grep for `|.*|.*|` pattern) +3. Document contains before/after code snippets (grep for "```python" and "```yaml") +4. Document references architecture-overview.md and terminal-yaml-definitions.md +5. No productivity multiplier claims (grep -i for "times faster\|multiplier\|X faster\|10x\|5x") +6. Developer positioned as agent (grep for "Giles\|developer" appearing as subject of decision verbs) +7. Author has reviewed and approved the draft (checkpoint) + + + +- Complete executive summary draft exists at `docs/explanations/ai-assisted-development.md` +- All 8 Phase 1 requirements (CONT-01 through CONT-06, ADVC-01, ADVC-02) are addressed in the document +- All metrics are verified against the live git repository +- Document follows all locked decisions from 01-CONTEXT.md +- Author has reviewed and approved the draft + + + +After completion, create `.planning/phases/01-document-content/01-01-SUMMARY.md` + From 492223904ab263b8d58b6fc4192678dda2ccc9c1 Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 13:02:59 +0000 Subject: [PATCH 11/12] feat(01-01): write AI-assisted development executive summary - Verify metrics against live git repository (29 terminal types, 154 commits, 128 by developer) - Draft complete executive summary with outcome-first narrative structure - Include verified metrics table, before/after code snippets, honest AI framing - Cross-reference architecture-overview.md, terminal-yaml-definitions.md, fastcs-epics-ioc.md - Document builder2ibek trajectory as evidence of maturing AI practice Co-Authored-By: Claude Opus 4.6 --- docs/explanations/ai-assisted-development.md | 128 +++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 docs/explanations/ai-assisted-development.md diff --git a/docs/explanations/ai-assisted-development.md b/docs/explanations/ai-assisted-development.md new file mode 100644 index 0000000..3b13194 --- /dev/null +++ b/docs/explanations/ai-assisted-development.md @@ -0,0 +1,128 @@ +# AI-Assisted Architectural Transformation of CATio + +A single developer, working with Claude as an AI pair programmer over approximately +two weeks, transformed CATio from a system requiring hand-written Python for each +terminal type into one driven entirely by YAML definitions. The result: 29 terminal +types across two packages, roughly 17,000 lines of Python, and over 14,000 lines of +YAML terminal definitions -- delivered through 154 non-merge commits between +mid-December 2025 and early February 2026. + +## The Transformation + +### Before + +CATio's original architecture required a dedicated Python class for every +EtherCAT (a real-time industrial fieldbus protocol) terminal type. The file +`catio_hardware.py` contained 20 such classes across 1,204 lines, each one +manually defining I/O attributes through repetitive `add_attribute()` calls. +Adding support for a new Beckhoff terminal meant writing 30 to 60 lines of +Python boilerplate -- copying an existing class, adjusting channel counts and +attribute names, and hoping the pattern stayed consistent. + +### After + +A single factory function, `get_terminal_controller_class()`, now reads YAML +definitions at runtime and generates FastCS controller classes dynamically. Terminal +types are described declaratively in `terminal_types.yaml`, and a companion +`catio_terminals` package provides a NiceGUI web editor for managing those +definitions. Adding a new terminal type means editing a YAML file -- or using +the GUI. + +### The Difference in Practice + +**Before:** one Python class per terminal type (20 classes, 1,204 lines) + +```python +class EL1004Controller(CATioTerminalController): + io_function = "4-channel digital input, 24V DC, 3ms filter" + num_channels = 4 + async def get_io_attributes(self): + for i in range(1, self.num_channels + 1): + self.add_attribute(f"DICh{i}Value", AttrR(datatype=Int(), ...)) +``` + +**After:** one YAML block per terminal type (29 types, generated from vendor XML) + +```yaml +EL1004: + description: 4Ch. Dig. Input 24V, 3ms + symbol_nodes: + - name_template: Channel {channel} + channels: 4 + access: Read-only + selected: true +``` + +## By the Numbers + +| Metric | Value | Quality Indicator | +|--------|-------|-------------------| +| Terminal types | 29 defined in YAML | Validated against Beckhoff ESI XML | +| Source files | 16 before, 56 after | Documented across 10 explanation pages | +| Python (fastcs_catio) | 11,424 lines across 19 files | Includes dynamic controller modules | +| Python (catio_terminals) | 6,256 lines across 31 files | GUI editor, XML parser, data models | +| YAML definitions | 14,276 lines | Machine-generated from vendor XML, human-editable | +| Legacy classes | 20 in catio_hardware.py | Retained as validated fallback for original set | +| Timeline | ~2 weeks core (Jan 22 -- Feb 5) | Within a broader Dec -- Feb development period | +| Commits | 128 by the developer, 154 total | Includes testing, documentation, and cleanup | + +The transformation touched both packages: `fastcs_catio` grew from a monolithic +controller file into a set of dynamic modules (`catio_dynamic_controller.py`, +`catio_dynamic_symbol.py`, `catio_dynamic_coe.py`, `catio_dynamic_types.py`), while +`catio_terminals` was built from scratch as a standalone GUI tool for editing and +validating terminal definitions. + +## How AI Accelerated Development + +Giles Knap, the project's developer, brought deep domain expertise in EtherCAT, +TwinCAT ADS (Automation Device Specification -- Beckhoff's communication protocol), +and the FastCS framework. He directed the architectural vision throughout: the +decision to move from hard-coded classes to YAML-driven generation, the design of +the terminal definition schema, and the choice to create a separate GUI package. Claude +served as a pair programmer, handling the bulk implementation under that direction. + +Concrete tasks Claude performed included: generating controller boilerplate for the +20 original terminal types, implementing protocol handler code for ADS communication, +parsing Beckhoff ESI XML files to extract terminal definitions into YAML, drafting +architecture documentation (commit `4dcc9d5`: 1,307 lines across 14 files), and +refactoring modules when complexity grew. The developer's workflow evolved during the +project -- he began writing feature documentation first, then directing Claude to +implement the code, using the docs as a specification. + +Two AI model tiers were used via a GitHub Copilot subscription: Claude Sonnet handled +primary implementation tasks, while Claude Opus was brought in for analysis and +refactoring of larger modules. This division was practical -- when `catio_hardware.py` +exceeded 12,000 lines during development, Sonnet struggled with the context window, +so Opus analysed the file and proposed a decomposition into the smaller dynamic modules +that exist today. + +This project was not the developer's first use of AI-assisted development. An earlier +project, builder2ibek, had explored AI pair programming for a simpler migration tool. +CATio's work predated the structured agent workflows and skill definitions that +builder2ibek later adopted, making it a more manual but foundational experience. The +trajectory from CATio to builder2ibek shows a practice maturing over successive +projects -- not a one-off experiment. + +## What This Means Going Forward + +The core payoff is maintainability. When new Beckhoff terminals enter the market, +adding support to CATio is now a configuration task rather than a coding task: add a +YAML block to `terminal_types.yaml` (or use the GUI editor), and the dynamic +controller factory handles the rest. This has already been demonstrated -- 10 +additional terminal types were added after the initial set with zero Python changes +to the dynamic controller code. + +For the project's long-term health, this means that contributors do not need to +understand the controller framework internals to add terminal support. The barrier +to entry dropped from "write a Python class that correctly implements the FastCS +attribute protocol" to "describe the terminal's channels and +CoE (CANopen over EtherCAT) parameters in YAML." + +## Learn More + +- [Architecture Overview](architecture-overview.md) -- full architecture with + diagrams showing the dynamic controller generation pipeline +- [Terminal YAML Definitions](terminal-yaml-definitions.md) -- complete YAML schema, + how to add new terminals, and GUI editor usage +- [FastCS EPICS IOC](fastcs-epics-ioc.md) -- dynamic controller generation details + and runtime attribute creation From 3e74b449eabe770d7a939aa0d0a9af138f43fb6e Mon Sep 17 00:00:00 2001 From: Giles Knap Date: Mon, 2 Mar 2026 15:41:40 +0000 Subject: [PATCH 12/12] add ai-assisted dev doc --- .planning/config.json | 2 +- docs/explanations.md | 1 + docs/explanations/ai-assisted-development.md | 182 +++++++------------ 3 files changed, 70 insertions(+), 115 deletions(-) diff --git a/.planning/config.json b/.planning/config.json index 4082c74..35d058b 100644 --- a/.planning/config.json +++ b/.planning/config.json @@ -15,4 +15,4 @@ }, "parallelization": true, "brave_search": false -} \ No newline at end of file +} diff --git a/docs/explanations.md b/docs/explanations.md index b36486a..86f6b44 100644 --- a/docs/explanations.md +++ b/docs/explanations.md @@ -15,4 +15,5 @@ explanations/coe-parameters.md explanations/dynamic-pdos.md explanations/useful_notes_ads_twincat.md explanations/decisions.md +explanations/ai-assisted-development.md ``` diff --git a/docs/explanations/ai-assisted-development.md b/docs/explanations/ai-assisted-development.md index 3b13194..2e66e57 100644 --- a/docs/explanations/ai-assisted-development.md +++ b/docs/explanations/ai-assisted-development.md @@ -1,122 +1,76 @@ # AI-Assisted Architectural Transformation of CATio -A single developer, working with Claude as an AI pair programmer over approximately -two weeks, transformed CATio from a system requiring hand-written Python for each -terminal type into one driven entirely by YAML definitions. The result: 29 terminal -types across two packages, roughly 17,000 lines of Python, and over 14,000 lines of -YAML terminal definitions -- delivered through 154 non-merge commits between -mid-December 2025 and early February 2026. - -## The Transformation - -### Before - -CATio's original architecture required a dedicated Python class for every -EtherCAT (a real-time industrial fieldbus protocol) terminal type. The file -`catio_hardware.py` contained 20 such classes across 1,204 lines, each one -manually defining I/O attributes through repetitive `add_attribute()` calls. -Adding support for a new Beckhoff terminal meant writing 30 to 60 lines of -Python boilerplate -- copying an existing class, adjusting channel counts and -attribute names, and hoping the pattern stayed consistent. - -### After - -A single factory function, `get_terminal_controller_class()`, now reads YAML -definitions at runtime and generates FastCS controller classes dynamically. Terminal -types are described declaratively in `terminal_types.yaml`, and a companion -`catio_terminals` package provides a NiceGUI web editor for managing those -definitions. Adding a new terminal type means editing a YAML file -- or using -the GUI. - -### The Difference in Practice - -**Before:** one Python class per terminal type (20 classes, 1,204 lines) - -```python -class EL1004Controller(CATioTerminalController): - io_function = "4-channel digital input, 24V DC, 3ms filter" - num_channels = 4 - async def get_io_attributes(self): - for i in range(1, self.num_channels + 1): - self.add_attribute(f"DICh{i}Value", AttrR(datatype=Int(), ...)) -``` - -**After:** one YAML block per terminal type (29 types, generated from vendor XML) - -```yaml -EL1004: - description: 4Ch. Dig. Input 24V, 3ms - symbol_nodes: - - name_template: Channel {channel} - channels: 4 - access: Read-only - selected: true -``` - -## By the Numbers - -| Metric | Value | Quality Indicator | -|--------|-------|-------------------| -| Terminal types | 29 defined in YAML | Validated against Beckhoff ESI XML | -| Source files | 16 before, 56 after | Documented across 10 explanation pages | -| Python (fastcs_catio) | 11,424 lines across 19 files | Includes dynamic controller modules | -| Python (catio_terminals) | 6,256 lines across 31 files | GUI editor, XML parser, data models | -| YAML definitions | 14,276 lines | Machine-generated from vendor XML, human-editable | -| Legacy classes | 20 in catio_hardware.py | Retained as validated fallback for original set | -| Timeline | ~2 weeks core (Jan 22 -- Feb 5) | Within a broader Dec -- Feb development period | -| Commits | 128 by the developer, 154 total | Includes testing, documentation, and cleanup | - -The transformation touched both packages: `fastcs_catio` grew from a monolithic -controller file into a set of dynamic modules (`catio_dynamic_controller.py`, -`catio_dynamic_symbol.py`, `catio_dynamic_coe.py`, `catio_dynamic_types.py`), while -`catio_terminals` was built from scratch as a standalone GUI tool for editing and -validating terminal definitions. +## Overview + +A single developer, working with GitHub Copilot as an AI pair programmer over two weeks, performed a radical re-architecture of the fastcs-CATio project. + +The major new features are as follows. + +### Terminal Definitions + +YAML files describing each terminal (EtherCAT slice) type replace individual python classes. The YAML is entirely autogenerated from XML published by Beckhoff. + +This allowed the replacement of hand coded python classes for 29 terminal types with automatic support for all possible terminal types from Beckhoff, including future types yet to be released. + +### Web GUI + +An entirely new Web GUI supports developers implementing CATio IOCs. + +The GUI allows them to import terminal types directly from Beckhoff. + +It then presents them with an overview of the configurable features of those terminals so they can choose settings and specify which symbols the IOC should monitor. + +This serves not only as a configuration tool, but as an educational tool for browsing the symbols exposed by terminals and understanding the supported modes for each terminal. + +### CATio Hardware Simulation + +The first task was to build a complete simulator that can clone any existing physical EtherCAT chain. + +This simulator allowed for the implementation of comprehensive system tests, allowing high confidence when verifying radical architectural changes. + +### Documentation + +Comprehensive documentation was added where none existed before. This includes tutorials to familiarize developers with the simulator and new GUI. See [Tutorials](../tutorials) ## How AI Accelerated Development -Giles Knap, the project's developer, brought deep domain expertise in EtherCAT, -TwinCAT ADS (Automation Device Specification -- Beckhoff's communication protocol), -and the FastCS framework. He directed the architectural vision throughout: the -decision to move from hard-coded classes to YAML-driven generation, the design of -the terminal definition schema, and the choice to create a separate GUI package. Claude -served as a pair programmer, handling the bulk implementation under that direction. - -Concrete tasks Claude performed included: generating controller boilerplate for the -20 original terminal types, implementing protocol handler code for ADS communication, -parsing Beckhoff ESI XML files to extract terminal definitions into YAML, drafting -architecture documentation (commit `4dcc9d5`: 1,307 lines across 14 files), and -refactoring modules when complexity grew. The developer's workflow evolved during the -project -- he began writing feature documentation first, then directing Claude to -implement the code, using the docs as a specification. - -Two AI model tiers were used via a GitHub Copilot subscription: Claude Sonnet handled -primary implementation tasks, while Claude Opus was brought in for analysis and -refactoring of larger modules. This division was practical -- when `catio_hardware.py` -exceeded 12,000 lines during development, Sonnet struggled with the context window, -so Opus analysed the file and proposed a decomposition into the smaller dynamic modules -that exist today. - -This project was not the developer's first use of AI-assisted development. An earlier -project, builder2ibek, had explored AI pair programming for a simpler migration tool. -CATio's work predated the structured agent workflows and skill definitions that -builder2ibek later adopted, making it a more manual but foundational experience. The -trajectory from CATio to builder2ibek shows a practice maturing over successive -projects -- not a one-off experiment. - -## What This Means Going Forward - -The core payoff is maintainability. When new Beckhoff terminals enter the market, -adding support to CATio is now a configuration task rather than a coding task: add a -YAML block to `terminal_types.yaml` (or use the GUI editor), and the dynamic -controller factory handles the rest. This has already been demonstrated -- 10 -additional terminal types were added after the initial set with zero Python changes -to the dynamic controller code. - -For the project's long-term health, this means that contributors do not need to -understand the controller framework internals to add terminal support. The barrier -to entry dropped from "write a Python class that correctly implements the FastCS -attribute protocol" to "describe the terminal's channels and -CoE (CANopen over EtherCAT) parameters in YAML." +### Background + +This was the developer's first production project using AI. They also had very little background in CATio or knowledge of the existing state of the project. + +### Assistance + +The AI pair programmer helped in multiple areas: + +- rapid understanding of a new codebase +- help in understanding Beckhoff TwinCAT, +- discussing ideas for radical architecture changes +- investigating in depth the huge and complex XML files from Beckhoff + - to discover what could be extracted from them (ans: all details of all terminal types) + - to work out how to parse them into simpler YAML for a specified subset of terminals + - validating understanding of the meaning of various symbols and modes in DLS terminal types +- help designing a Model-View-Presenter pattern GUI app +- help choosing the best python libraries for various capabilities +- heavy lifting on writing 12000 lines of python for the new GUI and dynamic classes + - the majority of code was written by AI during 1 week of iteration + - all the code was then reviewed in depth by the developer, taking one work day + - the primary feedback from the review was restructuring into smaller modules and occasional refactor of repeated code +- exemplary help in diagnosing and fixing any issues with code/docs/infrastructure/tools +- excellent support in learning how to use agents themselves +- generally being able to learn about any subject that you need help with +- +fastcs-catio was an initial foray into AI and represents early learning about prompting only (stage 1 below). Nevertheless there was a performance improvement of 10x or more over traditional practices. + +### Further Improvements + +AI skills can be categorized into 4 levels that build upon each other: + +1. Prompt engineering -- knowing how to frame your conversation with the AI. +1. Context engineering -- knowing how to keep the context lean, clean and focused on the current task. Keeping global instructions small, loading skills at the right time, distributing tasks to multiple agents. +1. Intent engineering -- understanding how to keep the agent on task and focused on your goal. Being extremely clear about your intent. +1. Spec engineering -- very clearly specifying the desired outcome so that it is possible to leave a team of agents running long term to iterate on plan-execute-verify until your spec is met. + +Since the fastcs-catio project, reasonable progress in 2 and 3 has been made in other projects. For example see [builder2ibek](https://github.com/epics-containers/builder2ibek) which uses skills and multi-agent workflows in order to convert traditional xmlBuilder IOCs to epics-containers. Learning these additional skills makes AI even more powerful and efficient. ## Learn More