diff --git a/AGENTS.md b/AGENTS.md index 6ff4e6f5..8add21a3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -199,7 +199,7 @@ This project interfaces with Beckhoff EtherCAT I/O terminals via the ADS protoco - **Testing with Hardware**: **NEVER** run `fastcs-catio ioc` commands yourself. Let the user run the IOC and report any errors back to you. The IOC requires network access to real hardware that may not be available or may have specific configuration requirements. -- **Terminal Definitions**: YAML files describing Beckhoff terminal types, their symbols, and CoE objects. See [docs/explanations/terminal-definitions.md](docs/explanations/terminal-definitions.md) for: +- **Terminal Definitions**: YAML files describing Beckhoff terminal types, their symbols, and CoE objects. See [docs/explanations/terminal-yaml-definitions.md](docs/explanations/terminal-yaml-definitions.md) for: - How to generate terminal YAML files using `catio-terminals` - Understanding ADS symbol nodes and index groups - The difference between XML-defined symbols and ADS runtime symbols (e.g., `WcState`) @@ -213,7 +213,7 @@ This project interfaces with Beckhoff EtherCAT I/O terminals via the ADS protoco - **catio-terminals**: GUI editor for terminal YAML files. Use `catio-terminals update-cache` to fetch Beckhoff XML definitions, then use `catio-terminals edit [filename]` to edit files with the GUI. -- **Terminal YAML Files Are Generated**: **NEVER manually edit** terminal YAML files in `src/catio_terminals/terminals/`. These files are generated from Beckhoff XML by the code in `src/catio_terminals/xml_parser.py` and `src/catio_terminals/xml_pdo.py`. If the YAML has incorrect values: +- **Terminal YAML Files Are Generated**: **NEVER manually edit** terminal YAML files in `src/catio_terminals/terminals/`. These files are generated from Beckhoff XML by the code in `src/catio_terminals/xml/`. If the YAML has incorrect values: 1. Fix the XML parsing code that generates the YAML 2. Regenerate the YAML using `uv run catio-terminals clean-yaml ` (default is src/catio_terminals/terminals/terminal_types.yaml) 3. Manual edits will be lost on next regeneration @@ -221,7 +221,7 @@ This project interfaces with Beckhoff EtherCAT I/O terminals via the ADS protoco **Special cases:** - Index groups default to 0xF031/0xF021 for standard I/O - Counter terminals (group_type="Measuring") use 0xF030/0xF020 instead - - Group-specific logic is in `process_pdo_entries()` in `xml_pdo.py` + - Group-specific logic is in `process_pdo_entries()` in `xml/pdo.py` ## Agent Skills diff --git a/README.md b/README.md index 2a8661ab..0b2077a4 100644 --- a/README.md +++ b/README.md @@ -3,12 +3,9 @@ [![PyPI](https://img.shields.io/pypi/v/fastcs-catio.svg)](https://pypi.org/project/fastcs-catio) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0) -# fastcs_catio +# fastcs-catio -Control system integration of EtherCAT I/O devices running under TwinCAT using pyads and FastCS - -This is where you should write a short paragraph that describes what your module does, -how it does it, and why people should use it. +CATio provides control system integration for Beckhoff EtherCAT I/O devices running under TwinCAT. It uses the ADS protocol to communicate with TwinCAT PLCs and exposes device data as EPICS Process Variables through the FastCS framework. Source | :---: | :---: @@ -17,16 +14,14 @@ Docker | `docker run ghcr.io/diamondlightsource/fastcs-catio:latest` Documentation | Releases | -This is where you should put some images or code snippets that illustrate -some relevant examples. If it is a library then you might put some -introductory code here: - -Some tips about using Claude Code[CLAUDE_NOTES.md]. +## Quick Start -```python -from fastcs_catio import __version__ +```bash +# Install +pip install fastcs-catio -print(f"Hello fastcs_catio {__version__}") +# Run the IOC +fastcs-catio ioc --target-ip 192.168.1.100 ``` diff --git a/docs/_static/custom.css b/docs/_static/custom.css index b09ecfbb..d7d9168f 100644 --- a/docs/_static/custom.css +++ b/docs/_static/custom.css @@ -10,6 +10,13 @@ height: auto !important; } +/* Simple diagrams - don't force full width, use natural size */ +.mermaid-simple .mermaid, +.mermaid-simple .mermaid svg { + width: auto !important; + max-width: 100% !important; +} + /* Make expanded/fullscreen mermaid diagrams prioritize width with vertical scroll */ .mermaid-modal, .mermaid-fullscreen, diff --git a/docs/explanations.md b/docs/explanations.md index d256934c..b36486a3 100644 --- a/docs/explanations.md +++ b/docs/explanations.md @@ -8,10 +8,8 @@ Explanations of how it works and why it works that way. explanations/architecture-overview.md explanations/fastcs-epics-ioc.md explanations/ads-client.md -explanations/api-decoupling.md explanations/nomenclature.md explanations/terminal-yaml-definitions.md -explanations/terminal-definitions.md explanations/composite-types.md explanations/coe-parameters.md explanations/dynamic-pdos.md diff --git a/docs/explanations/ads-client.md b/docs/explanations/ads-client.md index ab44def0..046fd573 100644 --- a/docs/explanations/ads-client.md +++ b/docs/explanations/ads-client.md @@ -50,13 +50,7 @@ TwinCAT maintains a routing table of authorized clients. CATio uses UDP messages 1. **Discover the target's AMS NetId**: Query the TwinCAT system for its network identity 2. **Register this client**: Add an entry to the routing table with authentication credentials -```{literalinclude} ../../src/fastcs_catio/client.py -:language: python -:start-at: class RemoteRoute -:end-before: def _get_route_info_as_bytes -``` - -The route registration includes: +The `RemoteRoute` class in [client.py](../../src/fastcs_catio/client.py) handles route management with these parameters: | Parameter | Purpose | |-----------|---------| @@ -72,14 +66,7 @@ Default TwinCAT credentials are typically `Administrator` / `1`. Production syst ### TCP Connection -Once routed, CATio opens a persistent TCP connection for ADS communication: - -```{literalinclude} ../../src/fastcs_catio/client.py -:language: python -:pyobject: AsyncioADSClient.connected_to -``` - -The connection uses Python's `asyncio.open_connection()` for non-blocking I/O, returning stream reader/writer pairs for bidirectional communication. +Once routed, CATio opens a persistent TCP connection for ADS communication. The `AsyncioADSClient.connected_to()` method in [client.py](../../src/fastcs_catio/client.py) uses Python's `asyncio.open_connection()` for non-blocking I/O, returning stream reader/writer pairs for bidirectional communication. ## ADS Commands @@ -142,13 +129,7 @@ ADS symbols provide named access to PLC variables, avoiding hard-coded index gro ### The AdsSymbol Structure -```{literalinclude} ../../src/fastcs_catio/devices.py -:language: python -:pyobject: AdsSymbol -:end-before: @property -``` - -Symbols carry type information (`dtype`) allowing CATio to correctly interpret binary data. The `group` and `offset` fields are used in ADS read/write commands. +The `AdsSymbol` dataclass in [devices.py](../../src/fastcs_catio/devices.py) represents ADS symbols. Symbols carry type information (`dtype`) allowing CATio to correctly interpret binary data. The `group` and `offset` fields are used in ADS read/write commands. ### Symbol-Based Access vs Direct Access @@ -201,7 +182,7 @@ The client uses string-based dispatch to route API calls: - `query("SYSTEM_TREE")` → calls `get_system_tree()` - `command("DEVICE_STATE", ...)` → calls `set_device_state(...)` -This pattern, while flexible, has tradeoffs discussed in [API Decoupling Analysis](api-decoupling.md). +This pattern, while flexible, has tradeoffs discussed in [API Decoupling Analysis](decisions/0003-api-decoupling-analysis.md). ### Key API Methods @@ -230,15 +211,8 @@ The client raises exceptions with meaningful messages when operations fail, allo ## Testing with MockADSServer -CATio includes a mock ADS server for testing without hardware: - -```{literalinclude} ../../tests/mock_server.py -:language: python -:pyobject: MockADSServer -:end-before: async def start -``` +CATio includes a mock ADS server for testing without hardware. The `MockADSServer` class in [mock_server.py](../../tests/mock_server.py) provides: -The mock server: - Accepts TCP connections on the standard ADS port - Parses AMS headers and dispatches to command handlers - Returns configurable mock responses @@ -250,4 +224,4 @@ This enables comprehensive testing of the client logic independent of real TwinC - [Architecture Overview](architecture-overview.md) - High-level system architecture - [FastCS EPICS IOC Implementation](fastcs-epics-ioc.md) - Details of the EPICS layer -- [API Decoupling Analysis](api-decoupling.md) - API design discussion +- [API Decoupling Analysis](decisions/0003-api-decoupling-analysis.md) - API design discussion diff --git a/docs/explanations/architecture-overview.md b/docs/explanations/architecture-overview.md index 678c75e2..a95dcb25 100644 --- a/docs/explanations/architecture-overview.md +++ b/docs/explanations/architecture-overview.md @@ -8,14 +8,19 @@ CATio is a Python-based control system integration for EtherCAT I/O devices runn ## High-Level Architecture Diagram ```mermaid -%%{init: {'theme': 'base', 'themeVariables': { 'fontSize': '14px'}}}%% flowchart TB clients["EPICS Clients / Control Systems"] subgraph fastcs["FastCS EPICS IOC Layer"] direction LR - server["CATioServerController"] --> device["CATioDeviceController
(EtherCAT Master)"] --> terminal["CATioTerminalController
(EK1100, EL3xxx, etc.)"] - fastcs_files["catio_controller.py
catio_hardware.py
catio_attribute_io.py"] + server["CATioServerController"] --> device["CATioDeviceController
(EtherCAT Master)"] --> terminal["Dynamic Terminal Controllers
(generated from YAML)"] + fastcs_files["catio_controller.py
catio_dynamic_controller.py
catio_dynamic_symbol.py
catio_dynamic_coe.py"] + end + + subgraph yamldef["Terminal Definitions"] + direction LR + yaml["terminal_types.yaml"] --> dynamic["get_terminal_controller_class()"] + runtime["runtime_symbols.yaml"] --> dynamic end subgraph bridge["CATio API Bridge"] @@ -36,6 +41,7 @@ flowchart TB end clients -->|"Channel Access / PVAccess"| fastcs + yamldef -->|"Controller classes"| fastcs fastcs -->|"CATioConnection API
(CATioFastCSRequest/Response)"| bridge bridge -->|"ADS Protocol (TCP/UDP)"| adslayer adslayer -->|"ADS/AMS Protocol
(TCP 48898, UDP 48899)"| twincat @@ -49,9 +55,31 @@ The top layer provides EPICS integration through the FastCS framework: - **CATioServerController**: Root controller representing the I/O server; manages TCP connections and device discovery - **CATioDeviceController**: Represents EtherCAT Master devices with their associated attributes -- **CATioTerminalController**: Represents individual EtherCAT slave terminals (couplers, I/O modules) +- **Dynamic Terminal Controllers**: Generated at runtime from YAML terminal definitions (see below) - **CATioControllerAttributeIO**: Handles attribute read/write operations through the API +### Dynamic Controller Generation + +Terminal controllers are generated dynamically from YAML definitions in `src/catio_terminals/terminals/`: + +- **`get_terminal_controller_class()`**: Factory function that creates controller classes on demand +- **`catio_dynamic_controller.py`**: Creates FastCS controller classes from YAML terminal type definitions +- **`catio_dynamic_symbol.py`**: Adds PDO symbol attributes (process data) to controllers +- **`catio_dynamic_coe.py`**: Adds CoE parameter attributes (configuration) to controllers +- **`catio_dynamic_types.py`**: Type conversion between TwinCAT, numpy, and FastCS types + +Controller classes are cached so only one class is created per terminal type: + +```python +from fastcs_catio.catio_dynamic_controller import get_terminal_controller_class + +# Get or create a controller class for a terminal type +controller_class = get_terminal_controller_class("EL1004") + +# Use it like any other controller class +controller = controller_class(name="MOD1", node=node) +``` + ### API Bridge Layer The middle layer provides a clean interface between FastCS and the ADS client: @@ -77,25 +105,44 @@ The bottom layer implements the TwinCAT ADS protocol: 2. **Route Addition**: Client machine is added to the TwinCAT server's routing table 3. **TCP Connection**: Establish persistent TCP connection for ADS communication 4. **I/O Introspection**: Query server for devices, slaves, and symbol information -5. **Controller Creation**: Build FastCS controller hierarchy matching hardware topology -6. **Attribute Registration**: Create EPICS PVs for each accessible parameter +5. **Dynamic Controller Creation**: Generate controller classes from YAML definitions based on discovered terminal types +6. **Attribute Registration**: Create EPICS PVs for each accessible parameter (symbols and CoE objects) ### Runtime Data Flow ```mermaid -flowchart TB - A["EPICS Client Request"] --> B["FastCS Attribute Access"] - B --> C["CATioControllerAttributeIO.update()"] - C --> D["CATioConnection.send_query()"] - D --> E["AsyncioADSClient.query() / command()"] - E --> F["API method dispatch (get_* / set_*)"] - F --> G["ADS Read/Write/ReadWrite commands"] - G --> H["TwinCAT Server Response"] - H --> I["Response propagation back to EPICS"] +sequenceDiagram + participant Client as EPICS Client + participant FastCS as FastCS Attribute + participant IO as CATioControllerAttributeIO + participant Conn as CATioConnection + participant ADS as AsyncioADSClient + participant TC as TwinCAT Server + + Client->>FastCS: Read/Write PV + FastCS->>IO: update() + IO->>Conn: send_query() + Conn->>ADS: query() / command() + ADS->>ADS: API method dispatch (get_* / set_*) + ADS->>TC: ADS Read/Write/ReadWrite + TC-->>ADS: Response + ADS-->>Conn: Response + Conn-->>IO: Response + IO-->>FastCS: Update attribute + FastCS-->>Client: PV updated ``` ## Key Design Decisions +### Dynamic Controller Generation + +Terminal controllers are generated from YAML definitions rather than explicit Python classes: + +- **Flexibility**: New terminal types can be added by editing YAML without code changes +- **Maintainability**: Single source of truth for terminal definitions in `terminal_types.yaml` +- **Runtime symbols**: Diagnostic symbols from the EtherCAT master defined separately in `runtime_symbols.yaml` +- **Selection**: Only symbols marked `selected: true` in YAML become FastCS attributes + ### Asynchronous Architecture The entire stack uses Python's `asyncio` for non-blocking I/O operations: @@ -112,8 +159,8 @@ Controllers form a tree structure mirroring the physical EtherCAT topology: IOServer └── IODevice (EtherCAT Master) ├── IOSlave (EK1100 Coupler) - │ ├── IOSlave (EL3xxx Input) - │ └── IOSlave (EL4xxx Output) + │ ├── IOSlave (EL3xxx Input) - DynamicEL3xxxController + │ └── IOSlave (EL4xxx Output) - DynamicEL4xxxController └── IOSlave (EK1101 Coupler) └── ... ``` @@ -137,6 +184,7 @@ CATio is configured through command-line parameters: ## See Also -- [FastCS EPICS IOC Implementation](fastcs-epics-ioc.md) - Detailed explanation of the EPICS layer -- [ADS Client Implementation](ads-client.md) - Detailed explanation of the ADS protocol layer -- [API Decoupling Analysis](api-decoupling.md) - Discussion of the API design and potential improvements +- [FastCS EPICS IOC Implementation](fastcs-epics-ioc.md) - Details of the EPICS layer +- [ADS Client Implementation](ads-client.md) - Details of the ADS protocol layer +- [Terminal YAML Definitions](terminal-yaml-definitions.md) - How to define terminal types in YAML +- [API Decoupling Analysis](decisions/0003-api-decoupling-analysis.md) - Discussion of the API design and potential improvements diff --git a/docs/explanations/composite-types.md b/docs/explanations/composite-types.md index aaa39356..dc25cb92 100644 --- a/docs/explanations/composite-types.md +++ b/docs/explanations/composite-types.md @@ -1,6 +1,6 @@ # Composite Types in TwinCAT -This document explains how TwinCAT generates composite type names and how they are implemented in the catio-terminals codebase. It complements the [Terminal Definitions](terminal-definitions.md) document. +This document explains how TwinCAT generates composite type names and how they are implemented in the catio-terminals codebase. It complements the [Terminal YAML Definitions](terminal-yaml-definitions.md) document. ## What Are Composite Types? @@ -212,6 +212,6 @@ When adding support for a new terminal type: ## Related Documentation -- [Terminal Definitions](terminal-definitions.md) - YAML file structure and symbol nodes +- [Terminal YAML Definitions](terminal-yaml-definitions.md) - YAML file structure and symbol nodes - [Beckhoff XML Format](../reference/beckhoff-xml-format.md) - ESI XML schema reference - [Architecture Overview](architecture-overview.md) - Overall system design diff --git a/docs/explanations/api-decoupling.md b/docs/explanations/decisions/0003-api-decoupling-analysis.md similarity index 98% rename from docs/explanations/api-decoupling.md rename to docs/explanations/decisions/0003-api-decoupling-analysis.md index 28446d73..c5669704 100644 --- a/docs/explanations/api-decoupling.md +++ b/docs/explanations/decisions/0003-api-decoupling-analysis.md @@ -444,6 +444,6 @@ These improvements would make the codebase more maintainable, testable, and easi ## See Also -- [Architecture Overview](architecture-overview.md) - High-level system architecture -- [FastCS EPICS IOC Implementation](fastcs-epics-ioc.md) - Details of the EPICS layer -- [ADS Client Implementation](ads-client.md) - Details of the ADS protocol layer +- [Architecture Overview](../architecture-overview.md) - High-level system architecture +- [FastCS EPICS IOC Implementation](../fastcs-epics-ioc.md) - Details of the EPICS layer +- [ADS Client Implementation](../ads-client.md) - Details of the ADS protocol layer diff --git a/docs/explanations/dynamic-pdos.md b/docs/explanations/dynamic-pdos.md index c9664e61..e2d639db 100644 --- a/docs/explanations/dynamic-pdos.md +++ b/docs/explanations/dynamic-pdos.md @@ -205,7 +205,7 @@ The data models for PDO groups are defined in [models.py](../../src/catio_termin ### XML Parsing -The `xml_pdo_groups.py` module handles parsing of PDO group definitions using two methods: +The `xml/pdo_groups.py` module handles parsing of PDO group definitions using two methods: **Method 1: AlternativeSmMapping (preferred)** diff --git a/docs/explanations/fastcs-epics-ioc.md b/docs/explanations/fastcs-epics-ioc.md index c6d4ad26..35c18d34 100644 --- a/docs/explanations/fastcs-epics-ioc.md +++ b/docs/explanations/fastcs-epics-ioc.md @@ -42,11 +42,7 @@ All CATio controllers inherit from `CATioController`, which extends the FastCS ` - References to corresponding hardware objects (`IOServer`, `IODevice`, or `IOSlave`) - Attribute grouping for organized PV naming -```{literalinclude} ../../src/fastcs_catio/catio_controller.py -:language: python -:start-at: class CATioController -:end-before: @property -``` +The `CATioController` class is defined in [catio_controller.py](../../src/fastcs_catio/catio_controller.py). It includes connection management, attribute registration, and the core interface for communicating with the ADS client. ### The Server Controller @@ -78,21 +74,46 @@ During initialization, the server controller queries the TwinCAT system and buil | `LinkStatus` | Network link health indicator | | `CrcErrorSum` | Accumulated CRC errors for this terminal | -## Hardware-Specific Controllers +## Dynamic Terminal Controllers + +Not all terminals are alike. A digital input module exposes different data than an analog output module. CATio handles this through dynamically generated controller classes based on YAML terminal definitions. + +### How Dynamic Generation Works + +When CATio discovers a terminal, it calls `get_terminal_controller_class(terminal_id)` from `catio_dynamic_controller.py`. This factory function: + +1. Looks up the terminal type (e.g., "EL3064") in `terminal_types.yaml` +2. Creates a controller class dynamically based on the YAML definition +3. Adds symbol attributes for process data (from `catio_dynamic_symbol.py`) +4. Adds CoE attributes for configuration parameters (from `catio_dynamic_coe.py`) +5. Caches the class for reuse + +The key modules involved: -Not all terminals are alike. A digital input module exposes different data than an analog output module. CATio handles this through specialized controller classes defined in [catio_hardware.py](../../src/fastcs_catio/catio_hardware.py). +| Module | Purpose | +|--------|---------| +| `catio_dynamic_controller.py` | Factory function and dynamic class creation | +| `catio_dynamic_symbol.py` | Adds PDO symbol attributes to controllers | +| `catio_dynamic_coe.py` | Adds CoE parameter attributes to controllers | +| `catio_dynamic_types.py` | Type conversion between TwinCAT, numpy, and FastCS | -The `SUPPORTED_CONTROLLERS` dictionary maps Beckhoff terminal type codes to their controller classes. When CATio discovers a terminal, it looks up the type (e.g., "EL3064") in this dictionary and instantiates the appropriate controller. +### Terminal YAML Definitions -Each terminal type exposes its specific attributes: +Each terminal type is defined in `src/catio_terminals/terminals/terminal_types.yaml` with: -| Controller Family | Terminal Types | Key Attributes | -|-------------------|----------------|----------------| -| `EL10xxController` | EL1004, EL1008, etc. | Digital input values | -| `EL20xxController` | EL2004, EL2008, etc. | Digital output values (read/write) | -| `EL30xxController` | EL3064, EL3102, etc. | Analog input values, scaling info | -| `EL40xxController` | EL4002, EL4132, etc. | Analog output values, range config | -| `ELM3xxxController` | ELM3004, ELM3602, etc. | High-precision measurements, oversampling | +- **Symbol nodes**: Process data accessible via ADS (inputs/outputs) +- **CoE objects**: Configuration parameters with subindices +- **Selection**: Only symbols with `selected: true` become attributes + +For example, a digital input terminal might expose: + +| Attribute Type | Source | Examples | +|----------------|--------|----------| +| PDO symbols | `symbol_nodes` in YAML | Input values, status bits | +| Runtime symbols | `runtime_symbols.yaml` | WcState, InfoData | +| CoE parameters | `coe_objects` in YAML | Filter settings, calibration | + +This approach allows adding new terminal types by editing YAML files without changing Python code. See [Terminal YAML Definitions](terminal-yaml-definitions.md) for details on the YAML format. ## The Attribute I/O System @@ -110,11 +131,7 @@ The update flow for a CATio attribute follows these steps: This indirection means attributes don't need to know ADS protocol details - they just specify their name and polling period. -```{literalinclude} ../../src/fastcs_catio/catio_attribute_io.py -:language: python -:start-at: class CATioControllerAttributeIO -:end-before: class -``` +The `CATioControllerAttributeIO` class in [catio_attribute_io.py](../../src/fastcs_catio/catio_attribute_io.py) implements this bridge between FastCS attributes and the ADS client API. ### Polling vs Notifications @@ -181,13 +198,7 @@ CATio controllers follow a specific lifecycle managed by FastCS: ## Testing Considerations -When writing tests for CATio controllers, you typically need to mock the ADS client layer. The `MockADSServer` class in the test suite simulates TwinCAT responses: - -```{literalinclude} ../../tests/mock_server.py -:language: python -:pyobject: MockADSServer -:end-before: async def start -``` +When writing tests for CATio controllers, you typically need to mock the ADS client layer. The `MockADSServer` class in [mock_server.py](../../tests/mock_server.py) simulates TwinCAT responses. This allows testing controller logic without real hardware by: @@ -199,4 +210,4 @@ This allows testing controller logic without real hardware by: - [Architecture Overview](architecture-overview.md) - High-level system architecture - [ADS Client Implementation](ads-client.md) - Details of the ADS protocol layer -- [API Decoupling Analysis](api-decoupling.md) - Discussion of the API design +- [API Decoupling Analysis](decisions/0003-api-decoupling-analysis.md) - API design discussion diff --git a/docs/explanations/terminal-definitions.md b/docs/explanations/terminal-definitions.md deleted file mode 100644 index 90ce164f..00000000 --- a/docs/explanations/terminal-definitions.md +++ /dev/null @@ -1,255 +0,0 @@ -# Terminal Type Definitions - -Terminal type definitions in CATio describe Beckhoff EtherCAT I/O terminals and their characteristics. These definitions are stored in YAML files organized by terminal class in `src/catio_terminals/terminals/`. - -## Purpose - -Terminal type definitions serve two purposes: - -1. **ADS Simulation**: The test ADS simulator uses these definitions to emulate terminal behavior and create accurate symbol tables -2. **FastCS Integration**: (Future) These definitions will be used to dynamically generate FastCS controller classes for each terminal type - -## Generating Terminal Definitions with catio-terminals - -The recommended way to create and maintain terminal definition files is using the `catio-terminals` GUI editor. This tool fetches terminal information from Beckhoff's XML descriptions (ESI files) and generates consistent YAML files. - -### Installation - -```bash -uv pip install -e ".[terminals]" -``` - -### Updating the XML Cache - -Before generating terminal definitions, update the local XML cache from Beckhoff's servers: - -```bash -# this command is also available in the terminal editor GUI -# you would only need to run it if ~/.cache/catio_terminals/ is outdated -catio-terminals update-cache -``` - -This downloads and parses the latest ESI XML files from Beckhoff, storing them in `~/.cache/catio_terminals/terminals_cache.json`. - -### Creating or Editing Terminal Files - -Launch the GUI editor: - -```bash -catio-terminals -``` - -Or open an existing file directly: - -```bash -catio-terminals path/to/terminals.yaml -``` - -### Workflow - -1. **Open or create** a YAML file when prompted -2. **Add terminals** by clicking "Add Terminal" and searching Beckhoff's catalog -3. **Select symbols** to include from the available PDO entries shown in the XML -4. **Save** the file - only selected symbols are written to YAML - -The editor merges your YAML selections with the XML definitions, ensuring that: -- Symbols are derived from Beckhoff's official XML descriptions (TxPdo/RxPdo entries) -- Any symbols in the YAML that don't exist in the XML are dropped with a warning -- New symbols from XML updates can be discovered and added - -## File Organization - -Terminal definitions are organized by functional class: - -- `bus_couplers.yaml` - EtherCAT couplers and extensions (EK1100, EK1110, etc.) -- `digital_input.yaml` - Digital input terminals (EL1004, EL1014, EL1084, etc.) -- `digital_output.yaml` - Digital output terminals (EL2004, EL2024, EL2809, etc.) -- `counter.yaml` - Counter and frequency input terminals (EL1502, etc.) -- `analog_input.yaml` - Analog input terminals (EL3004, EL3104, EL3602, etc.) -- `analog_output.yaml` - Analog output terminals (EL4004, EL4134, etc.) -- `power_supply.yaml` - Power supply and system terminals (EL9410, EL9512, etc.) - -## Terminal Definition Structure - -Each terminal type definition contains: - -### Identity - -CANopen identity information that uniquely identifies the terminal: - -```yaml -identity: - vendor_id: 2 # Vendor ID (2 = Beckhoff) - product_code: 196882514 # Product code (decimal) - revision_number: 1048576 # Revision number (decimal) -``` - -### Symbol Nodes - -Symbol nodes define the Process Data Objects (PDOs) available on the terminal. These are extracted from the TxPdo (inputs) and RxPdo (outputs) entries in Beckhoff's XML descriptions. - -```yaml -symbol_nodes: - - name_template: "Value {channel}" - index_group: 61472 # ADS index group (0xF020 = 61472) - size: 2 # Data size in bytes - ads_type: 2 # ADS data type (2=INT, 3=DINT, etc.) - type_name: "INT" # Data type name - channels: 4 # Number of channels - access: "Read-only" # Access mode - fastcs_name: "Value{channel}" # PascalCase name for FastCS -``` - -### Symbol Node Properties - -| Property | Description | -|----------|-------------| -| `name_template` | Name pattern supporting `{channel}` placeholder | -| `index_group` | ADS index group (61472=0xF020 for inputs, 61488=0xF030 for outputs) | -| `size` | Data size in bytes | -| `ads_type` | ADS data type code | -| `type_name` | Data type name (BOOL, INT, DINT, UINT, etc.) | -| `channels` | Number of channels (for multi-channel terminals) | -| `access` | Access mode: "Read-only" or "Read/Write" | -| `fastcs_name` | PascalCase name used for FastCS attribute naming | - -### ADS Index Groups - -| Index Group | Hex | Name | Purpose | -|-------------|-----|------|---------| -| 61472 | 0xF020 | RWIB | Input bytes (read/write) | -| 61473 | 0xF021 | RWIX | Input bits (read/write) | -| 61488 | 0xF030 | RWOB | Output bytes (read/write) | -| 61489 | 0xF031 | RWOX | Output bits (read/write) | - -### CoE Objects - -CANopen over EtherCAT objects for terminal configuration (optional): - -```yaml -coe_objects: - - index: 0x8000 - name: "Settings" - type_name: "USINT" - bit_size: 8 - access: "rw" -``` - -### Group Type - -The terminal's functional group from the XML: - -```yaml -group_type: AnaIn # AnaIn, AnaOut, DigIn, DigOut, etc. -``` - -## ADS Runtime Symbols vs XML Definitions - -When introspecting real hardware via ADS, you may see additional symbols that don't appear in the XML definitions, such as `WcState^WcState` or `InputToggle`. These are **ADS runtime symbols** added by the EtherCAT master, not terminal-specific PDO data. - -### What are Runtime Symbols? - -The `WcState^WcState` and similar symbols come from the **ADS runtime symbol table** when you query actual hardware. These are **EtherCAT Working Counter** status bits that the TwinCAT/ADS runtime adds dynamically to indicate whether each terminal is responding correctly on the EtherCAT bus. - -### Known Runtime Symbols - -TwinCAT adds these standardized diagnostic symbols to terminals: - -| Symbol | Type | Size | Description | -|--------|------|------|-------------| -| `WcState^WcState` | BIT | 1 bit | Working Counter State - indicates if terminal is communicating properly on the EtherCAT bus | -| `InfoData^State` | UINT16 | 2 bytes | Device state information (EtherCAT state machine) | -| `InputToggle` | BIT | 1 bit | Toggles on each EtherCAT cycle to indicate data freshness | - -These symbols are: -- **Mostly standardized** - similar across terminal types, but not all terminals have all symbols -- **Generated at runtime** - not defined in ESI XML files -- **Diagnostic in nature** - used for monitoring bus health, not process data - -### Why aren't they in the XML? - -Runtime symbols are **not** in the Beckhoff XML terminal description files because: - -1. The XML files describe the **static hardware capabilities** of each terminal type (PDO mappings, CoE objects) -2. The WcState symbols are **runtime diagnostics** added by the EtherCAT master when it configures the bus -3. These symbols are added dynamically based on bus configuration, not terminal capabilities - -### Runtime Symbols Configuration - -Runtime symbols are defined in `src/catio_terminals/config/runtime_symbols.yaml` using a schema similar to terminal symbol nodes, with additional filtering capabilities: - -```yaml -runtime_symbols: - - name_template: WcState^WcState - index_group: 61473 # 0xF021 - RWIX (input bits) - size: 0 - ads_type: 33 # BIT - type_name: BIT - channels: 1 - access: Read-only - fastcs_name: WcstateWcstate - description: Working Counter State - indicates bus communication status - # Filtering options: - group_blacklist: - - Coupler # Couplers may not have WcState -``` - -#### Filtering Options - -Each runtime symbol can specify which terminals it applies to: - -| Field | Description | -|-------|-------------| -| `whitelist` | Only apply to these specific terminal IDs (e.g., `["EL3004", "EL3104"]`) | -| `blacklist` | Exclude these specific terminal IDs | -| `group_whitelist` | Only apply to terminals in these groups (e.g., `["AnaIn", "DigIn"]`) | -| `group_blacklist` | Exclude terminals in these groups (e.g., `["Coupler"]`) | - -If no filters are specified, the symbol applies to all terminals. Whitelist takes precedence over blacklist. - -### Data Sources - -| Source | Availability | Contains | -|--------|--------------|----------| -| XML (ESI files) | Downloadable from Beckhoff, scrapable | Static terminal capabilities, PDO mappings | -| Runtime Symbols | Defined in `runtime_symbols.yaml` | Dynamic diagnostics from EtherCAT master | - -The runtime symbols are documented in Beckhoff InfoSys but the content uses heavy JavaScript rendering that makes it difficult to scrape programmatically. The relevant documentation pages are: - -- [TwinCAT I/O Variables](https://infosys.beckhoff.com/content/1033/tc3_io_intro/1257993099.html) -- [EtherCAT Diagnosis](https://infosys.beckhoff.com/content/1033/ethercatsystem/2469122443.html) - -### Key Differences - -| Source | Contains | Examples | -|--------|----------|----------| -| XML (ESI files) | Static terminal capabilities, PDO mappings | `Value {channel}`, `Status__Error {channel}` | -| Runtime Symbols | Dynamic diagnostics from EtherCAT master | `WcState^WcState`, `InputToggle`, `InfoData^State` | - -### Handling in catio-terminals - -The catio-terminals editor separates XML-defined symbols from runtime symbols: - -- **XML symbols**: Derived from Beckhoff ESI files, shown in the terminal editor -- **Runtime symbols**: Loaded from `runtime_symbols.yaml`, filtered per terminal based on whitelist/blacklist rules -- When merging YAML files with XML data, symbols not in XML or runtime symbols are dropped with a warning - -The symbol expansion logic in `src/fastcs_catio/symbols.py` handles runtime symbol types when reading from actual hardware. - -## Adding New Terminal Types - -To add a new terminal type: - -1. Launch `catio-terminals` and open the appropriate YAML file -2. Click "Add Terminal" and search for the terminal ID (e.g., "EL3104") -3. Select the symbols you want to include from the XML definition -4. Save the file - -The terminal will be automatically available to the ADS simulator and FastCS integration. - -## See Also - -- [ADS Client Architecture](ads-client.md) -- [Architecture Overview](architecture-overview.md) -- Terminal type source files: `src/catio_terminals/terminals/` -- catio-terminals documentation: `src/catio_terminals/README.md` diff --git a/docs/explanations/terminal-yaml-definitions.md b/docs/explanations/terminal-yaml-definitions.md index 78ae2274..47760094 100644 --- a/docs/explanations/terminal-yaml-definitions.md +++ b/docs/explanations/terminal-yaml-definitions.md @@ -11,24 +11,19 @@ For background on the underlying Beckhoff technologies, see: ## Overview Terminal definitions are YAML files that describe: +- **Composite types** - Bit field definitions for packed status/control bytes - **Terminal identity** - Vendor ID, product code, revision - **Symbol nodes** - Process data (I/O values) accessible via ADS -- **CoE objects** - Configuration parameters (optional) -- **Runtime symbols** - Diagnostic symbols added by the EtherCAT master +- **CoE objects** - Configuration parameters with subindices +- **Runtime symbols** - Diagnostic symbols added by the EtherCAT master (defined separately) ## File Organization -Terminal definitions live in `src/catio_terminals/terminals/`, organized by function: +Terminal definitions live in `src/catio_terminals/terminals/`, organized as follows: | File | Terminal Types | |------|----------------| -| `analog_input.yaml` | EL3004, EL3104, EL3602, etc. | -| `analog_output.yaml` | EL4004, EL4134, etc. | -| `digital_input.yaml` | EL1004, EL1014, EL1084, etc. | -| `digital_output.yaml` | EL2004, EL2024, EL2809, etc. | -| `counter.yaml` | EL1502, EL5101, etc. | -| `bus_couplers.yaml` | EK1100, EK1110, etc. | -| `power_supply.yaml` | EL9410, EL9512, etc. | +| `terminal_types.yaml` | All terminals currently in use at DLS | Supporting configuration files in `src/catio_terminals/config/`: @@ -38,9 +33,20 @@ Supporting configuration files in `src/catio_terminals/config/`: ## Terminal Definition Structure -### Basic Example +### Top-Level Structure ```yaml +composite_types: + InputBits: + description: 'Bit fields: Input' + ads_type: 65 + size: 1 + members: [] + bit_fields: + - name: Input + bit: 0 + # ... more composite types + terminal_types: EL3104: description: "4-channel Analog Input +/-10V 16-bit" @@ -50,13 +56,36 @@ terminal_types: revision_number: 1048576 group_type: AnaIn symbol_nodes: - - name_template: "AI Standard Channel {channel}.Value" - index_group: 61472 - type_name: INT - channels: 4 - access: Read-only - fastcs_name: "AiStandardChannel{channel}Value" - coe_objects: [] + # ... symbols + coe_objects: + # ... CoE objects +``` + +### Composite Types Section + +Composite types define packed bit fields for status and control bytes: + +```yaml +composite_types: + Status__Underrange_6Bits: + description: 'Bit fields: Status__Underrange, Status__Overrange, Status__Error, + Status__Sync error, Status__TxPDO State, Status__TxPDO Toggle' + ads_type: 65 + size: 1 + members: [] + bit_fields: + - name: Status__Underrange + bit: 0 + - name: Status__Overrange + bit: 1 + - name: Status__Error + bit: 2 + - name: Status__Sync error + bit: 3 + - name: Status__TxPDO State + bit: 4 + - name: Status__TxPDO Toggle + bit: 5 ``` ### Identity Section @@ -78,22 +107,31 @@ Symbol nodes define the ADS symbols for process data. Each symbol maps to memory ```yaml symbol_nodes: - - name_template: "AI Standard Channel {channel}.Value" - index_group: 61472 # 0xF020 - input data - type_name: INT - channels: 4 + - name_template: CNT Inputs Channel {channel} + index_group: 61488 + type_name: Status__Output functions enabled_7Bits + channels: 2 + access: Read-only + fastcs_name: cnt_inputs_channel_{channel} + selected: false + - name_template: CNT Inputs.Counter value + index_group: 61488 + type_name: UDINT + channels: 1 access: Read-only - fastcs_name: "AiStandardChannel{channel}Value" + fastcs_name: cnt_inputs_counter_value + selected: true ``` | Field | Description | |-------|-------------| | `name_template` | Symbol name pattern. Use `{channel}` for multi-channel terminals | | `index_group` | ADS index group (see table below) | -| `type_name` | Data type - primitive (INT, BOOL, etc.) | +| `type_name` | Data type - primitive (INT, BOOL, UDINT) or composite type name | | `channels` | Number of channels using this pattern | | `access` | `Read-only` (inputs) or `Read/Write` (outputs) | -| `fastcs_name` | PascalCase name for FastCS attributes | +| `fastcs_name` | Snake_case name for FastCS attributes | +| `selected` | Whether this symbol is included in FastCS controllers | **ADS Index Groups:** @@ -104,18 +142,57 @@ symbol_nodes: | 61488 | 0xF030 | Output bytes (RxPDO - controller to terminal) | | 61489 | 0xF031 | Output bits | +### CoE Objects + +CoE (CANopen over EtherCAT) objects define configuration parameters with subindices: + +```yaml +coe_objects: + - index: 32768 # 0x8000 in decimal + name: CNT Settings Ch.1 + type_name: DT8000 + bit_size: 128 + access: rw + subindices: + - subindex: 1 + name: Enable function to set output + type_name: BOOL + bit_size: 1 + access: rw + default_data: '00' + fastcs_name: enable_function_to_set_output_idx8000 + - subindex: 17 + name: Switch on treshold value + type_name: UDINT + bit_size: 32 + access: rw + default_data: '00000000' + fastcs_name: switch_on_treshold_value_idx8000 + fastcs_name: cnt_settings_ch_1_idx8000 +``` + +| Field | Description | +|-------|-------------| +| `index` | CoE index (decimal) | +| `name` | Human-readable name | +| `type_name` | Data type for the object | +| `bit_size` | Total size in bits | +| `access` | `ro` (read-only) or `rw` (read-write) | +| `subindices` | List of subindex definitions | +| `fastcs_name` | FastCS attribute name | + ### Computed Properties The `size` and `ads_type` fields are computed from `type_name`: ```yaml # You write: -- name_template: "Value {channel}" - type_name: INT +- name_template: "Counter value" + type_name: UDINT # The system computes: -# size: 2 (bytes) -# ads_type: 2 (ADS type code for INT) +# size: 4 (bytes) +# ads_type: 19 (ADS type code for UDINT) ``` Type mappings are defined in `src/catio_terminals/ads_types.py`. @@ -182,8 +259,8 @@ Downloads Beckhoff's ESI XML files to `~/.cache/catio_terminals/`. ### Edit Terminal Files ```bash -catio-terminals # Create new file -catio-terminals terminals.yaml # Edit existing file +catio-terminals edit # Create new file +catio-terminals edit terminals.yaml # Edit existing file ``` ### Workflow @@ -220,8 +297,8 @@ The test simulator uses terminal definitions to: ### FastCS Controller Generation -(Future) Terminal definitions will generate FastCS controllers: -- Each symbol becomes a FastCS attribute +Terminal definitions generate FastCS controllers: +- Each symbol with `selected: true` becomes a FastCS attribute - Channel templates expand to numbered attributes ### catio-terminals UI diff --git a/docs/how-to/get-all-pdos-in-chain.md b/docs/how-to/get-all-pdos-in-chain.md deleted file mode 100644 index edf07508..00000000 --- a/docs/how-to/get-all-pdos-in-chain.md +++ /dev/null @@ -1,394 +0,0 @@ -# How to Get All PDOs in an EtherCAT Chain - -This guide explains how to discover the actual PDO (Process Data Object) configuration of terminals at runtime, which is necessary when the ESI XML cannot reliably predict the symbol structure. - -The original intention of the YAML based terminal type descriptions created using `catio-terminals` (see [Terminal YAML Definitions](../explanations/terminal-yaml-definitions.md)) was to provide a complete mapping of all PDOs for each terminal type, generating this information from the Beckhoff ESI XML files. - -However, some terminals have dynamic PDO configurations that depend on project settings in TwinCAT, making it impossible to determine the exact PDO layout from XML alone. This document outlines the extent of the problem by highlighting those terminals used at Diamond Light Source that require runtime discovery. - -## Current Status - -At present the YAML definitions will include all of the dynamic PDOs. However, these are grouped by mode and the editor allows you to select which mode you are using. This in turn only selects the PDOs from that mode. - -This means that changing the mode on TwinCAT requires that you change the mode in the YAML file and restart the IOC. - -Please note that at present all terminals of a given type must be in the same mode for this to work as there is no Terminal instance level mode selection. This is a a planned improvement. - -## Overview - -Some EtherCAT terminals have **dynamic PDO configurations** that depend on: -- TwinCAT project settings -- Selected operation mode -- Oversampling factor -- PDO assignment choices - -For these terminals, you must query the TwinCAT symbol table at runtime rather than relying solely on XML-derived definitions. - -## Terminals in the Test Configuration - -The terminals listed here are the complete set used at DLS at the time of writing (see [DLS Terminals Reference](../reference/DLS_terminals.md) for details). - -The terminals are categorized by type, with notes on which require runtime discovery. - -| Terminal | Description | PDO Type | Notes | -|----------|-------------|----------|-------| -| EK1100 | EtherCAT Coupler | None | Bus coupler, no process data | -| EK1122 | 2 Port EtherCAT Junction | None | Junction module, no process data | -| EL1014 | 4Ch Digital Input 24V, 10us | Static | Fixed 4 BOOL inputs | -| EL1084 | 4Ch Digital Input 24V, 3ms, negative | Static | Fixed 4 BOOL inputs | -| EL1124 | 4Ch Digital Input 5V, 10us | Static | Fixed 4 BOOL inputs | -| EL1502 | 2Ch Up/Down Counter 24V, 100kHz | **Dynamic Type 1** | Multiple PDO configurations (per-channel vs combined) | -| EL2024 | 4Ch Digital Output 24V, 2A | Static | Fixed 4 BOOL outputs | -| EL2024-0010 | 4Ch Digital Output 24V, 2A (variant) | Static | Fixed 4 BOOL outputs | -| EL2124 | 4Ch Digital Output 5V, 20mA | Static | Fixed 4 BOOL outputs | -| EL2502 | 2Ch PWM Output 24V | Static | Fixed 2 UINT PWM outputs | -| EL2595 | 1Ch LED Constant Current | Static | Status + multi-field control outputs | -| EL2612 | 2Ch Relay Output CO | Static | Fixed 2 BOOL relay outputs | -| EL2624 | 4Ch Relay Output NO | Static | Fixed 4 BOOL relay outputs | -| EL3104 | 4Ch Analog Input +/-10V Diff | **Dynamic Type 2** | Standard vs Compact PDO selection | -| EL3124 | 4Ch Analog Input 4-20mA Diff | **Dynamic Type 2** | Standard vs Compact PDO selection | -| EL3202 | 2Ch Analog Input PT100 (RTD) | Static | Fixed RTD Inputs per channel | -| EL3202-0010 | 2Ch Analog Input PT100 (RTD, variant) | Static | Fixed RTD Inputs per channel | -| EL3314 | 4Ch Analog Input Thermocouple | Static | Fixed TC Inputs + optional CJ compensation outputs | -| EL3356-0010 | 1Ch Resistor Bridge, 16bit High Precision | **Dynamic Type 3** | Multiple PDO formats (INT32, Real, Standard, Compact) | -| EL3602 | 2Ch Analog Input +/-10V Diff, 24bit | Static | Fixed AI Inputs per channel | -| EL3702 | 2Ch Analog Input +/-10V Oversample | **Dynamic Type 4** | 1-100 samples per channel | -| EL4134 | 4Ch Analog Output +/-10V, 16bit | Static | Fixed AO Output per channel | -| EL4732 | 2Ch Analog Output +/-10V Oversample | **Dynamic Type 4** | Configurable sample count | -| EL9410 | E-Bus Power Supply (Diagnostics) | Static | Fixed 2 status BOOLs (Us, Up undervoltage) | -| EL9505 | Power Supply Terminal 5V | Static | Fixed status (Power OK, Overload) | -| EL9510 | Power Supply Terminal 10V | Static | Fixed status (Power OK, Overload) | -| EL9512 | Power Supply Terminal 12V | Static | Fixed status (Power OK, Overload) | -| ELM3704-0000 | 4Ch Universal Analog Input, 24bit | **Dynamic Type 4** | Oversampling (1-100), data format (INT16/INT32/REAL), per-channel config | -| EP2338-0002 | 8Ch Digital I/O 24V, 0.5A | Static | Fixed 8 inputs + 8 outputs | -| EP2624-0002 | 4Ch Relay Output NO | Static | Fixed 4 BOOL relay outputs | -| EP3174-0002 | 4Ch Analog Input configurable | **Dynamic Type 2** | Standard vs Compact PDO selection | -| EP3204-0002 | 4Ch Analog Input PT100 (RTD) | Static | Fixed RTD Inputs per channel | -| EP3314-0002 | 4Ch Analog Input Thermocouple | Static | Fixed TC Inputs + optional CJ compensation outputs | -| EP4174-0002 | 4Ch Analog Output configurable | Static | Fixed AO Outputs per channel | -| EP4374-0002 | 2Ch AI + 2Ch AO configurable | **Dynamic Type 2** | Standard vs Compact PDO selection for inputs | - -## Terminals Requiring Runtime Discovery - -### Type 1: Counter Terminals with Configuration Variants (EL1502) - -#### EL1502 - Up/Down Counter - -**Problem:** The ESI XML defines multiple PDO configurations: - -| Configuration | PDO Names | Symbol Pattern | -|---------------|-----------|----------------| -| Per-channel | `CNT Inputs Channel 1`, `CNT Inputs Channel 2` | `CNT Inputs Channel {n}.Counter value` | -| Combined | `CNT Inputs` | `CNT Inputs.Counter value` | - -The YAML file includes **both** configurations, but only one will be active in TwinCAT. - -**Runtime Discovery:** -```python -# Query actual symbols for an EL1502 -symbols = await client.get_all_symbols() -el1502_symbols = [symbol for name, symbol in symbols[device_id].items() if "EL1502" in name] - -# Check which configuration is active -has_per_channel = any("Channel 1" in s.name for s in el1502_symbols) -has_combined = any("CNT Inputs.Counter value" in s.name for s in el1502_symbols) -``` - -### Type 2: Analog Input with Standard/Compact PDO Selection (EL3104, EL3124, EP3174-0002, EP4374-0002) - -**Problem:** These analog input terminals offer two PDO formats per channel: - -| Format | PDO Name Pattern | Contents | -|--------|------------------|----------| -| Standard | `AI Standard Channel {n}` | Status word (underrange, overrange, limits, error) + Value | -| Compact | `AI Compact Channel {n}` | Value only (no status) | - -The TwinCAT project selects one format per channel. Both are defined in XML as optional. - -**Runtime Discovery:** -```python -# Query actual symbols for analog input terminals -symbols = await client.get_all_symbols() -ai_symbols = [s for name, symbol in symbols[device_id].items() if "AI" in name] - -# Determine which format is active -has_standard = any("Standard" in s.name for s in ai_symbols) -has_compact = any("Compact" in s.name for s in ai_symbols) - -if has_standard: - print("Using Standard PDO format (status + value)") -elif has_compact: - print("Using Compact PDO format (value only)") -``` - -### Type 3: Multi-Format Terminals (EL3356-0010) - -**Problem:** The EL3356-0010 offers multiple PDO formats for measurement data: - -| Format | PDO Name | Data Type | -|--------|----------|-----------| -| INT32 | `RMB Value (INT32)` | DINT (32-bit signed) | -| Real | `RMB Value (Real)` | REAL (32-bit float) | -| Standard Ch1/Ch2 | `AI Standard Channel {n}` | Status + DINT value | -| Compact Ch1/Ch2 | `AI Compact Channel {n}` | DINT value only | - -Additionally, it has optional `RMB Status`, `RMB Timestamp`, and `RMB Control` PDOs. - -**Runtime Discovery:** -```python -# Query actual symbols for EL3356 -symbols = await client.get_all_symbols() -el3356_symbols = [s for name, symbol in symbols[device_id].items() if "EL3356" in name] - -# Determine which value format is active -has_int32 = any("RMB Value (INT32)" in s.name for s in el3356_symbols) -has_real = any("RMB Value (Real)" in s.name for s in el3356_symbols) -has_standard = any("AI Standard" in s.name for s in el3356_symbols) -has_compact = any("AI Compact" in s.name for s in el3356_symbols) - -# Check for optional features -has_timestamp = any("Timestamp" in s.name for s in el3356_symbols) -has_control = any("RMB Control" in s.name for s in el3356_symbols) -``` - -### Type 4: Oversampling Terminals (EL3702, EL4732, ELM3704-0000) - -#### EL3702 - 2Ch Oversampling Analog Input - -**Problem:** The number of samples per channel is configurable (1-100). The XML only shows a template with one sample. - -| Oversampling | Samples per Channel | Total Symbols | -|--------------|---------------------|---------------| -| 1x (default) | 1 | 2 values | -| 10x | 10 | 20 values | -| 100x | 100 | 200 values | - -**Runtime Discovery:** -```python -# Query actual symbols for an EL3702 -symbols = await client.get_all_symbols() -el3702_ch1 = [s for name, symbol in symbols[device_id].items() - if "EL3702" in name and "Ch1" in name and "Value" in name] - -oversampling_factor = len(el3702_ch1) -print(f"EL3702 configured for {oversampling_factor}x oversampling") -``` - -#### EL4732 - 2Ch Oversampling Analog Output - -**Problem:** Similar to EL3702, the sample count is configurable. The output PDO contains arrays of values. - -**Runtime Discovery:** -```python -# Query actual symbols for an EL4732 -symbols = await client.get_all_symbols() -el4732_symbols = [s for name, symbol in symbols[device_id].items() if "EL4732" in name and "Value" in name] - -# Count array elements to determine oversampling factor -sample_count = len(el4732_symbols) -print(f"EL4732 configured for {sample_count} samples per cycle") -``` - -#### ELM3704-0000 - 4Ch Universal Analog Input with Oversampling - -**Problem:** This is the most complex terminal. Each of the 4 channels can be independently configured with: - -1. **Data format**: INT16, INT32 (DINT), or REAL32 -2. **Oversampling factor**: 1, 2, 4, 5, 8, 10, 16, 20, 25, 32, 40, 50, 64, 80, or 100 samples -3. **Optional PDOs**: Status, Timestamp, Control, Cold Junction Temperature - -The XML defines 192 TxPDOs (48 per channel) covering all combinations! - -| PDO Pattern | Example | Description | -|-------------|---------|-------------| -| `PAI Status Channel {n}` | `PAI Status Channel 1` | Status byte per channel | -| `PAI Samples {count} Channel {n}` | `PAI Samples 10 Channel 1` | DINT array with count samples | -| `PAI Samples16 {count} Channel {n}` | `PAI Samples16 10 Channel 1` | INT array with count samples | -| `PAI SamplesR32 {count} Channel {n}` | `PAI SamplesR32 10 Channel 1` | REAL array with count samples | -| `PAI Timestamp Channel {n}` | `PAI Timestamp Channel 1` | 64-bit timestamp | - -**Runtime Discovery:** -```python -# Query actual symbols for ELM3704 -symbols = await client.get_all_symbols() -elm3704_symbols = [s for name, symbol in symbols[device_id].items() if "ELM3704" in name] - -for ch in range(1, 5): - ch_symbols = [s for s in elm3704_symbols if f"Channel {ch}" in s.name] - - # Determine data format - if any("SamplesR32" in s.name for s in ch_symbols): - data_format = "REAL32" - elif any("Samples16" in s.name for s in ch_symbols): - data_format = "INT16" - elif any("Samples " in s.name for s in ch_symbols): # Note space to avoid Samples16 - data_format = "INT32" - else: - data_format = "Unknown" - - # Determine oversampling factor from PDO name - import re - for s in ch_symbols: - match = re.search(r'Samples\d* (\d+) Channel', s.name) - if match: - oversample = int(match.group(1)) - break - else: - oversample = 1 - - print(f"Channel {ch}: {data_format} format, {oversample}x oversampling") -``` - -## How to Query All PDOs at Runtime - -### Step 1: Get Symbol Table Info - -```python -from fastcs_catio.client import CatioClient -from fastcs_catio._constants import IndexGroup - -# Request symbol table metadata -response = await client._ads_command( - AdsReadRequest( - index_group=IndexGroup.ADSIGRP_SYM_UPLOADINFO2, # 0xF00F - index_offset=0, - read_length=24, - ) -) -symbol_count = ... # Parse from response -table_length = ... # Parse from response -``` - -### Step 2: Download Full Symbol Table - -```python -# Fetch all symbol definitions -response = await client._ads_command( - AdsReadRequest( - index_group=IndexGroup.ADSIGRP_SYM_UPLOAD, # 0xF00B - index_offset=0, - read_length=table_length, - ) -) -# Parse symbol entries from response.data -``` - -### Step 3: Use the High-Level API - -The `CatioClient` provides a simpler interface: - -```python -async with CatioClient(target_ip="192.168.1.100") as client: - # Discover EtherCAT devices - await client.get_ethercat_devices() - - # Get all symbols from the symbol table - all_symbols = await client.get_all_symbols() - - # Symbols are keyed by device ID - for device_id, symbols in all_symbols.items(): - for name, symbol in symbols.items(): - print(f"{name}: {symbol.type_name} @ {symbol.index_group:#x}:{symbol.index_offset}") -``` - -## Recommendations - -1. **For static terminals** (EK1100, EK1122, EL1014, EL1084, EL1124, EL2024, EL2024-0010, EL2124, EL2502, EL2595, EL2612, EL2624, EL3202, EL3202-0010, EL3314, EL3602, EL4134, EL9410, EL9505, EL9510, EL9512, EP2338-0002, EP2624-0002, EP3204-0002, EP3314-0002, EP4174-0002): - - Use XML-derived YAML definitions directly - - PDO structure is fixed and predictable - -2. **For Type 1 counter terminals with configuration variants** (EL1502): - - Query symbol table to determine per-channel vs combined configuration - - YAML includes both; only one will be active - -3. **For Type 2 Standard/Compact selectable terminals** (EL3104, EL3124, EP3174-0002, EP4374-0002): - - Query symbol table at startup to determine which format is active - - YAML should define both formats; reconcile with actual symbols - -4. **For Type 3 multi-format terminals** (EL3356-0010): - - Query symbol table to determine active value format (INT32, Real, Standard, Compact) - - Check for optional features (Timestamp, Control) - -5. **For Type 4 oversampling terminals** (EL3702, EL4732, ELM3704-0000): - - Always query symbol table - sample count is runtime-configurable - - ELM3704 additionally requires checking data format per channel - - Use symbol naming patterns to extract configuration - -6. **For unknown terminals**: Always query the symbol table to discover available PDOs - -## See Also - -- [Useful Notes on ADS and TwinCAT](../explanations/useful_notes_ads_twincat.md) - Details on ADS access methods and XML limitations -- [Terminal YAML Definitions](../explanations/terminal-yaml-definitions.md) - How terminal definitions are structured - -## Appendix: Identifying Dynamic PDOs in ESI XML - -The need for runtime discovery can often be identified by inspecting the `VendorSpecific` section of the Beckhoff ESI XML file for the terminal. - -### The `` Tag - -For many terminals (Type 2, Type 3, and Type 4 above), the XML explicitly defines mutually exclusive PDO configurations using the `` tag within the `` vendor-specific section. - -**Example from EL3104 (Type 2):** - -```xml - - - - Standard - - #x1a00 - #x1a02 - ... - - - - Compact - - #x1a01 - #x1a03 - ... - - - - -``` - -**Example from EL3356 (Type 3):** - -In Type 3 terminals, this mechanism is used to switch between completely different data types (e.g., Integer vs Float) for the same process data. - -```xml - - - - Standard (INT32) - - #x1a00 - #x1a01 - - - - Standard (REAL) - - #x1a00 - #x1a02 - - - - -``` - -This structure indicates that the terminal has exclusive modes that fundamentally change which PDOs (and thus which data types and offsets) are assigned to Sync Manager 3 (Inputs). - -### Multiple PDOs for the Same Function - -For terminals like the EL1502 (Type 1), the indication is subtler. You will find multiple PDO definitions that cover overlapping functionality, and the CoE objects for PDO assignment (0x1C12 for RxPDOs, 0x1C13 for TxPDOs) will have a default value but allow modification. - -**Example from EL1502:** -The XML defines three TxPDO maps for inputs: -- `0x1A00`: CNT TxPDO-Map Ch.1 (Channel 1 only) -- `0x1A01`: CNT TxPDO-Map Ch.2 (Channel 2 only) -- `0x1A02`: CNT TxPDO-Map (Both channels combined) - -All three target the input memory, but TwinCAT will only allow one valid combination (either 0x1A00 + 0x1A01 OR 0x1A02) to be assigned to the Sync Manager at any time. The presence of "Combined" versus "Split" PDOs in the XML is a strong indicator of this dynamic behavior. diff --git a/docs/how-to/use-terminal-editor.md b/docs/how-to/use-terminal-editor.md index 2b87c530..a083b7f3 100644 --- a/docs/how-to/use-terminal-editor.md +++ b/docs/how-to/use-terminal-editor.md @@ -66,20 +66,22 @@ The editor has three main sections: ## Adding a Terminal 1. Click **"Add Terminal"** button in the header -2. Choose one of two methods: - - **Method 1: Search Beckhoff Catalog** - - Enter search terms in the search box - - Click "Search" - - Browse results and click "Add" on your chosen terminal - - The terminal information will be populated automatically - - **Method 2: Create Manually** - - Scroll to the bottom of the dialog - - Enter Terminal ID (e.g., "EL4004") - - Enter Description - - Click "Create Manually" - - Edit the default values as needed +2. The dialog shows a list of available terminals with search and filtering: + + **Filtering by Type** + - Use the "Filter by Type" dropdown to narrow results by terminal category (Digital Input, Analog Output, etc.) + - The filter automatically selects the category of your currently selected terminal + + **Searching** + - Type in the search box to filter terminals - results update automatically as you type + - Press Enter or just continue typing to refine the search + - The status line shows total matches, already-added count, and available count + + **Adding Terminals** + - Click **"Add"** on any available terminal to add it to your configuration + - Already-added terminals appear grayed out + - Use **"Add All"** to add all currently visible available terminals at once + - Click **"Close"** when finished ## Editing Terminal Details @@ -173,9 +175,17 @@ The `catio_terminals` package consists of several modules: - **models.py**: Pydantic data models for YAML structure - **beckhoff.py**: Client for fetching Beckhoff terminal information -- **xml_cache.py**: Caching system for Beckhoff XML files -- **xml_parser.py**: Parser for Beckhoff ESI XML files -- **app.py**: Main NiceGUI application +- **xml/**: Subpackage for Beckhoff ESI XML parsing + - **cache.py**: Caching system for downloaded XML files + - **parser.py**: Main XML parsing utilities + - **pdo.py**: PDO entry parsing + - **pdo_groups.py**: PDO group (AlternativeSmMapping) parsing + - **coe.py**: CoE object parsing + - **catalog.py**: Terminal catalog browsing +- **ui_app.py**: Main NiceGUI application +- **ui_components/**: UI component modules (tree view, details panes) +- **ui_dialogs/**: Dialog modules (file, terminal, confirmation dialogs) +- **service_*.py**: Service layer modules (file, config, terminal operations) - **__main__.py**: Entry point for the application ## Troubleshooting @@ -190,7 +200,8 @@ The `catio_terminals` package consists of several modules: - Check disk space - View error in notification -### Beckhoff Search Not Working -- The app downloads and caches Beckhoff XML files on first use -- Check internet connectivity +### Beckhoff Terminal Search Not Working +- Run `catio-terminals update-cache` to download and cache Beckhoff XML files +- Check internet connectivity during cache update - View cache status in `~/.cache/catio_terminals/` +- If the terminal list is empty, the cache needs to be updated diff --git a/docs/plans/documentation-plan.md b/docs/plans/documentation-plan.md new file mode 100644 index 00000000..605b0600 --- /dev/null +++ b/docs/plans/documentation-plan.md @@ -0,0 +1,74 @@ +# Documentation Plan + +This document outlines remaining documentation tasks and suggested agent prompts for improving the documentation. + +## Outstanding Tasks + +### High Priority + +- [ ] **Add Getting Started tutorial**: Create a step-by-step guide for first-time users covering installation, connection to TwinCAT, and running the IOC +- [ ] **Document CoE parameter write support**: If/when implemented, document how to write CoE configuration parameters + +### Medium Priority + +- [ ] **Add troubleshooting guide**: Common connection issues, ADS errors, and their solutions +- [ ] **Document notification subscriptions**: How to subscribe to high-frequency data updates using ADS notifications +- [ ] **Add configuration reference**: Document all CLI options and environment variables + +### Low Priority + +- [ ] **Add example scripts**: Python examples showing common use cases +- [ ] **Expand DLS terminals reference**: Add more detail on each terminal type's capabilities + +## Agent Prompts for Documentation Improvement + +Use these prompts to get AI assistance with documentation tasks: + +### Architecture and Design + +``` +Review the architecture-overview.md and suggest improvements to make the Mermaid diagrams clearer and more informative. +``` + +``` +Compare fastcs-epics-ioc.md and ads-client.md for consistency in terminology and cross-references. +``` + +### Terminal Definitions + +``` +Load Beckhoff XML skill and help me document a new terminal type [terminal-id]. +``` + +``` +Review terminal-yaml-definitions.md and identify any information that is out of date with the current code. +``` + +### Code Documentation + +``` +Review the docstrings in src/fastcs_catio/client.py and suggest improvements for clarity and completeness. +``` + +### User Guides + +``` +Write a troubleshooting guide covering common ADS connection errors and their solutions. +``` + +``` +Create a getting-started tutorial that walks through connecting to a TwinCAT PLC and reading analog input values. +``` + +## Documentation Structure + +The documentation follows the [Diátaxis](https://diataxis.fr/) framework: + +| Section | Purpose | Files | +|---------|---------|-------| +| **Tutorials** | Learning-oriented | `tutorials/installation.md` | +| **How-to** | Task-oriented | `how-to/` directory | +| **Explanations** | Understanding-oriented | `explanations/` directory | +| **Reference** | Information-oriented | `reference/` directory, `_api/` | + +When adding new documentation, place it in the appropriate section based on its purpose. diff --git a/docs/plans/simulator-symbol-alignment.md b/docs/plans/simulator-symbol-alignment.md deleted file mode 100644 index a8975bd7..00000000 --- a/docs/plans/simulator-symbol-alignment.md +++ /dev/null @@ -1,232 +0,0 @@ ---- -orphan: true ---- - -# Plan: Align Simulator Symbols with Real Hardware - -**Created:** 2026-01-27 -**Updated:** 2026-01-28 -**Status:** ✅ **Complete** - Simulator symbol count matches hardware (550 symbols) -**Related files:** -- [tests/ads_sim/ethercat_chain.py](../../tests/ads_sim/ethercat_chain.py) - Symbol generation logic -- [tests/ads_sim/server_config.yaml](../../tests/ads_sim/server_config.yaml) - Device/slave configuration -- [tests/diagnose_hardware.py](../../tests/diagnose_hardware.py) - Hardware comparison tool - -## Problem Summary - -The ADS simulator now generates symbols that exactly match real TwinCAT hardware: -1. ~~Symbol naming convention~~ ✅ Fixed -2. ~~Missing device-level symbols~~ ✅ Fixed -3. ~~Missing per-terminal WcState symbols~~ ✅ Fixed -4. ~~Index group assignments~~ ✅ Fixed -5. ~~Extra/different symbols per terminal type~~ ✅ Fixed (resolved by fixes 1-4, 6) -6. ~~Missing SyncUnits global symbol~~ ✅ Fixed - -| Metric | Simulator (Before) | Simulator (After) | Hardware | -|--------|-----------|-----------|----------| -| Total Symbols | 1091 | 550 ✅ | 550 | -| Naming Format | `TIID^Device 1 (EtherCAT)^Term X^...` | `Term X (type).Channel Y` ✅ | `Term X (type).Channel Y` | -| Device Symbols | 0 | 8 ✅ | 8 | -| Global Symbols | 0 | 1 ✅ | 1 (SyncUnits) | -| Per-Terminal Runtime | 0 | 140+ ✅ | 140+ (WcState, InputToggle) | - ---- - -## Issues and Tasks - -### Issue 1: Symbol Naming Convention -**Priority:** High -**Status:** [x] Complete (2026-01-28) - -**Problem:** -- Simulator used: `TIID^Device 1 (EtherCAT)^Term 4 (EL2024)^Channel 1` -- Hardware uses: `Term 10 (EL2024).Channel 1` - -**Solution:** -Updated `SymbolDefinition.expand_symbols()` in `ethercat_chain.py` to: -- Remove `TIID^Device {device_id} (EtherCAT)^` prefix -- Change `^` separator to `.` separator -- Use format: `{terminal_name}.{symbol_name}` - -**Files modified:** -- `tests/ads_sim/ethercat_chain.py` - `expand_symbols()` method - ---- - -### Issue 2: Missing Device-Level Symbols -**Priority:** High -**Status:** [x] Complete (2026-01-28) - -**Problem:** -Real hardware exposes EtherCAT master device symbols that simulator didn't generate. - -**Solution:** -Added `get_device_symbols()` method to `EtherCATDevice` class that returns 8 device-level symbols: -- `Device 1 (EtherCAT).Inputs.Frm0State` -- `Device 1 (EtherCAT).Inputs.Frm0WcState` -- `Device 1 (EtherCAT).Inputs.Frm0InputToggle` -- `Device 1 (EtherCAT).Inputs.SlaveCount` -- `Device 1 (EtherCAT).Inputs.DevState` -- `Device 1 (EtherCAT).Outputs.Frm0Ctrl` -- `Device 1 (EtherCAT).Outputs.Frm0WcCtrl` -- `Device 1 (EtherCAT).Outputs.DevCtrl` - -**Note:** Used `ADS_TYPE_BIT` (33) instead of `ADS_TYPE_UINT16` (18) because client's `symbol_lookup` doesn't handle UINT16. See [todo.md](todo.md) for follow-up task. - -**Files modified:** -- `tests/ads_sim/ethercat_chain.py` - Added `get_device_symbols()` method, updated `get_all_symbols()` - ---- - -### Issue 3: Missing Per-Terminal WcState Symbols -**Priority:** Medium -**Status:** [x] Complete (2026-01-28) - -**Problem:** -Each terminal on real hardware has WcState symbols that simulator didn't generate. - -**Solution:** -Integrated with `catio_terminals` runtime symbols system: -- Load `RuntimeSymbolsConfig` from `src/catio_terminals/config/runtime_symbols.yaml` -- Apply runtime symbols to each slave based on terminal type and group -- Symbols added: `WcState.WcState`, `WcState.InputToggle`, `InfoData.State` - -Runtime symbols are filtered by terminal group (DigIn, DigOut, AnaIn, etc.) as defined in the YAML config. - -**Note:** `InfoData.State` uses `ADS_TYPE_UINT16` which client doesn't handle - these symbols are generated but filtered by client. See [todo.md](todo.md). - -**Files modified:** -- `tests/ads_sim/ethercat_chain.py`: - - Added `_load_runtime_symbols()` method to `EtherCATChain` - - Added `group_type` field to `TerminalType` - - Updated `EtherCATSlave.get_symbols()` to include runtime symbols - - Updated `total_symbol_count` to filter unhandled ADS types - ---- - -### Issue 4: Index Group Assignments -**Priority:** Medium -**Status:** [x] Complete (2026-01-28) - -**Problem:** -Simulator was using wrong index groups. Hardware uses: -- `0xF020` (61472) - Device-level inputs (Frm0State, SlaveCount, DevState) -- `0xF021` (61473) - Terminal OUTPUT channels (EL2024, etc.) -- `0xF030` (61488) - (Not observed in hardware) -- `0xF031` (61489) - Terminal INPUT channels (EL1014, etc.) and WcState symbols - -**Solution:** -Fixed default index group assignments in XML parser: -- TxPdo (inputs from device) → `0xF031` (61489) -- RxPdo (outputs to device) → `0xF021` (61473) -- WcState runtime symbols → `0xF031` (61489) - -**Files modified:** -- `src/catio_terminals/xml_pdo.py` - Fixed line 404: Changed `0xF020/0xF030` to `0xF031/0xF021` -- `src/catio_terminals/config/runtime_symbols.yaml` - Fixed WcState and InputToggle to use 61489 -- Regenerated `src/catio_terminals/terminals/terminal_types.yaml` with corrected values - ---- - -### Issue 5: EL1502 Counter Terminal Symbol Mismatch -**Priority:** Medium -**Status:** [x] Complete (2026-01-28) - -**Problem:** -Simulator was generating extra symbols per terminal that hardware didn't have. - -**Root Cause:** -Issues 1-4 and 6 collectively resolved this: -- Issue 1 fixed naming to match hardware format -- Issue 2 added missing device-level symbols -- Issue 3 added per-terminal WcState symbols -- Issue 4 corrected index group assignments -- Issue 6 added the global SyncUnits symbol - -With all these fixes, the simulator now generates exactly the same symbols as hardware with no extras. - -**Verification:** -```bash -$ tests/diagnose_hardware.py --compare tests/ads_sim/server_config.yaml -Hardware Symbols: 550 -Simulator Total: 550 -Difference: +0 -✓ Hardware matches simulator -``` - -**Files modified:** -- Same files as Issues 1-4, 6 (no additional changes needed) - ---- - -### Issue 6: SyncUnits Symbol -**Priority:** Low -**Status:** [x] Complete (2026-01-28) - -**Problem:** -Hardware has a `SyncUnits._default_._unreferenced_.WcState.WcState` symbol that simulator didn't generate. - -**Solution:** -Added SyncUnits as a global runtime symbol in `runtime_symbols.yaml`: -- Single global symbol (not per-terminal) -- Index group: `0xF031` (61489) -- Offset: `0x00002FB0` (matches hardware) -- Type: BIT (ADS_TYPE_BIT = 33) - -Global runtime symbols are filtered out from per-terminal application via `is_global` field in `RuntimeSymbol` model. - -**Files modified:** -- `src/catio_terminals/config/runtime_symbols.yaml` - Added SyncUnits global symbol definition -- `src/catio_terminals/models.py` - Added `is_global` field to `RuntimeSymbol`, updated `applies_to_terminal()` -- `tests/ads_sim/ethercat_chain.py` - Updated `get_all_symbols()` to add global runtime symbols - ---- - -## Testing Strategy - -✅ **Complete** - All issues resolved and verified. - -Testing approach used: -1. Ran `./tests/diagnose_hardware.py --compare tests/ads_sim/server_config.yaml` against real hardware -2. Compared symbol counts and verified exact match (550 symbols) -3. Verified all symbol names match hardware format -4. Confirmed system tests pass with no regressions - -## Verification Commands - -```bash -# Compare simulator against hardware in real-time -./tests/diagnose_hardware.py --ip 172.23.242.42 \ - --compare tests/ads_sim/server_config.yaml - -# Expected output: -# Hardware Symbols: 550 -# Simulator Total: 550 -# Difference: +0 -# ✓ Hardware matches simulator -``` - -## Progress Log - -| Date | Issue | Action | Result | -|------|-------|--------|--------| -| 2026-01-27 | - | Initial analysis and plan created | Identified 6 issues | -| 2026-01-28 | 1 | Updated `expand_symbols()` naming format | Symbols now use `Term.Symbol` format | -| 2026-01-28 | 2 | Added `get_device_symbols()` to EtherCATDevice | 8 device-level symbols added | -| 2026-01-28 | 3 | Integrated runtime symbols from catio_terminals | WcState symbols added (140+ new symbols) | -| 2026-01-28 | - | Fixed relative import in server.py | Tests pass | -| 2026-01-28 | - | Updated `total_symbol_count` to filter unhandled types | Test assertion fixed | -| 2026-01-28 | 4 | Fixed index group assignments in XML parser | TxPdo→0xF031, RxPdo→0xF021, regenerated YAML | -| 2026-01-28 | 6 | Added SyncUnits global runtime symbol | 1 global symbol added | -| 2026-01-28 | 5 | All fixes combined resolved symbol mismatch | Symbol count now matches: 550 = 550 ✅ | -| 2026-01-28 | - | Added `--compare` flag to diagnose_hardware.py | Real-time comparison tool | -| 2026-01-28 | - | **Plan Complete** | **Simulator matches hardware exactly** | - ---- - -## Notes - -- The `diagnose_hardware.py` script can be used to compare simulator and hardware outputs -- Symbol alignment is important for tests in `test_system.py` to pass against real hardware -- The simulator should produce symbols that match hardware format exactly for proper testing -- Client's `symbol_lookup` limitation documented in [todo.md](todo.md) - doesn't handle `ADS_TYPE_UINT16` diff --git a/docs/plans/todo.md b/docs/plans/todo.md index 01e93936..267f9bc7 100644 --- a/docs/plans/todo.md +++ b/docs/plans/todo.md @@ -2,43 +2,32 @@ Outstanding tasks and improvements for CATio. ---- - -## 1. Client symbol_lookup should handle ADS_TYPE_UINT16 +## Client symbol_lookup should handle ADS_TYPE_UINT16 **Priority:** Medium **Status:** Not Started ### Background -The client's `symbol_lookup` function in `src/fastcs_catio/symbols.py` only handles a limited set of ADS data types: +The `symbol_lookup` function in `src/fastcs_catio/symbols.py` only handles these ADS data types: - `ADS_TYPE_BIT` (33) - `ADS_TYPE_BIGTYPE` (65) - `ADS_TYPE_UINT8` (17) -Symbols with other ADS types (e.g., `ADS_TYPE_UINT16 = 18`) are silently ignored with a warning log message. +Symbols with other types (e.g., `ADS_TYPE_UINT16 = 18`) are silently ignored. ### Problem Real TwinCAT hardware returns symbols with `ADS_TYPE_UINT16` for: -- Device-level symbols: `Device 1 (EtherCAT).Inputs.Frm0State`, `SlaveCount`, `DevState`, etc. -- Runtime symbols: `InfoData.State` (device state from EtherCAT state machine) - -These symbols are currently filtered out by the client, meaning they cannot be read or monitored. - -### Current Workaround - -In the simulator (`tests/ads_sim/ethercat_chain.py`): -1. Device-level symbols use `ADS_TYPE_BIT` (33) instead of `ADS_TYPE_UINT16` (18) -2. `total_symbol_count` property filters out symbols with unhandled ADS types to match client behavior +- Device-level symbols: `Device 1 (EtherCAT).Inputs.Frm0State`, `SlaveCount`, `DevState` +- Runtime symbols: `InfoData.State` (EtherCAT state machine) ### Proposed Fix -Add handling for `ADS_TYPE_UINT16` (and potentially other integer types) in `symbol_lookup`: +Add handling for `ADS_TYPE_UINT16` in `symbol_lookup`: ```python -# In src/fastcs_catio/symbols.py, add case for UINT16: case AdsDataType.ADS_TYPE_UINT16: symbols.append( AdsSymbol( @@ -56,18 +45,6 @@ case AdsDataType.ADS_TYPE_UINT16: ) ``` -### Files to Modify - -- `src/fastcs_catio/symbols.py` - Add cases for `ADS_TYPE_UINT16`, `ADS_TYPE_INT16`, etc. - -### After Implementation - -1. Update simulator device-level symbols to use correct `ADS_TYPE_UINT16` -2. Remove `ADS_TYPE_BIT` workaround from `get_device_symbols()` in `tests/ads_sim/ethercat_chain.py` -3. Update `total_symbol_count` to include UINT16 symbols -4. Verify with real hardware that UINT16 symbols are readable - -### Related +### Files -- Simulator symbol alignment plan: [simulator-symbol-alignment.md](simulator-symbol-alignment.md) -- Runtime symbols config: `src/catio_terminals/config/runtime_symbols.yaml` +- `src/fastcs_catio/symbols.py` - Add cases for UINT16 and other integer types diff --git a/docs/reference.md b/docs/reference.md index 7f5d1501..3bb08127 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -11,8 +11,8 @@ reference/DLS_terminals reference/beckhoff-xml-format reference/ads-symbols-and-coe reference/ethercat-composite-types +plans/documentation-plan plans/todo -plans/simulator-symbol-alignment genindex Release Notes ``` diff --git a/docs/reference/beckhoff-xml-format.md b/docs/reference/beckhoff-xml-format.md index 4829cf33..e6902a87 100644 --- a/docs/reference/beckhoff-xml-format.md +++ b/docs/reference/beckhoff-xml-format.md @@ -207,7 +207,7 @@ CANopen over EtherCAT objects provide configuration parameters: ## Mapping to Terminal YAML -The XML parser (`xml_parser.py`) transforms ESI data into terminal YAML: +The XML parser (`xml/parser.py` and related modules) transforms ESI data into terminal YAML: | XML Element | YAML Field | |-------------|------------| @@ -236,5 +236,5 @@ These must be obtained from a live TwinCAT system. ## Related Documentation -- [Terminal Definitions](../explanations/terminal-definitions.md) - YAML file structure +- [Terminal YAML Definitions](../explanations/terminal-yaml-definitions.md) - YAML file structure - [catio-terminals README](https://github.com/DiamondLightSource/fastcs-catio/tree/main/src/catio_terminals) - Tool usage diff --git a/src/catio_terminals/terminals/README.md b/src/catio_terminals/terminals/README.md index 864e3992..6e6a648d 100644 --- a/src/catio_terminals/terminals/README.md +++ b/src/catio_terminals/terminals/README.md @@ -11,7 +11,7 @@ For complete documentation on terminal definitions, including: - ADS runtime symbols vs XML definitions - Adding new terminal types -See the main documentation: [Terminal Type Definitions](../../../docs/explanations/terminal-definitions.md) +See the main documentation: [Terminal YAML Definitions](../../../docs/explanations/terminal-yaml-definitions.md) ## Quick Reference