diff --git a/docs/docs/platform/platform_manager.md b/docs/docs/platform/platform_manager.md deleted file mode 100644 index 194dc985a396c..0000000000000 --- a/docs/docs/platform/platform_manager.md +++ /dev/null @@ -1,177 +0,0 @@ -# PlatformManager - -## Introduction - -The objective of PlatformManager is to generalize low level device setup and -management across platforms. This avoids platform specific customized software -logic. All the Platform specific information should be encoded in configuration -files, and the same generic software should run on all platforms going forward. - -## Functions of Interest - -- **Inventory Management**: A switch system could have different FRUs. - PlatformManager is responsible for building the inventory of the FRUs in a - switch. PlatformManager needs to monitor in real time what FRU has been - removed or inserted into the switch. It provides an interface to software - components within and outside of PlatformManager about the FRUs and the FRU - EEPROM contents. -- **Dynamic Device Management**: When an FRU has been detected by the Inventory - Management, Dynamic Device Management will load the necessary kmods and enable - communication with the FRU. PlatformManager will also ensure that the FRU - devices get consistent naming within each platform irrespective of how it is - plugged in and across reboots. - -## Components - -- **Platform Specific Configuration (provided by Vendors):** It has information - about the FRUs, I2C devices and how they can be accessed. The configuration - format is defined here. See here for a sample config. -- **Platform Agnostic Software (developed by Meta):** Meta will write and - maintain the software. The source code is here. - -## Workflow - -### On Initialization - -- Determine Platform Type using dmidecode set by BIOS. Details are in BMC Lite - Specifications. -- Load the platform specific PlatformManager Config -- Discover the devices (by polling present registers or reading sysfs paths) - based on the config and load the necessary kmods. - -### Periodically - -- Detect devices by listening to present polling/interrupts (by polling for now, - since interrupts are not full-fledged) -- On device detection, use the PlatformManager Config to set up the device and - load the necessary kmods. - -## Usage - -- PlatformManager will be used at provisioning/boot-up and be executed before - any FRU/EEPROM/FW access. -- PlatformManager will be run in steady state of the switch to facilitate - dynamic detection of devices. - -## Vendor Expectations - -- Configuration: Vendor will provide the platform specific configuration - according to the specification in fboss repository. - - **Note:** The [specification](https://github.com/facebook/fboss/blob/main/fboss/platform/platform_manager/platform_manager_config.thrift) - contains more thorough descriptions of PlatformManager terminology along - with implementation details. -- Handoff: Vendor will run PlatformManager as part of the handoff. The running - and testing instructions will be provided in fboss repository. -- PCI Device IDs: All PCI devices (e.g., FPGA) should have unique Vendor ID and - Device ID in the PCI configuration space. -- Dmidecode Setup: The platform type should be obtainable through dmidecode, per - BMC-lite specification document -- BSP: The BSP should follow the BSP requirements specification to enable device - discovery and setup. We expect one set of kmods per vendor. - ---- - -## Modeling Requirements - -### PMUnit - -Unit used for modeling in PlatformManager. It typically matches with a FRU, but -not always. PMUnit and FRU terminologies are used interchangeably in this -document. PMUnit and FRU differences will be explicitly called out when they are -not interchangeable. - -### EEPROM - -It should store content in Meta EEPROM V6 format. - -### IDPROM - -- Any EEPROM which is used to identify the FRU/PMUnit type is called IDPROM. -- The IDPROM in the FRU MUST be connected in one of the following ways: - - Directly to an incoming I2C bus from the parent FRU. - - Directly to the CPU’s SMBus I2C Controller. -- The Product Name field in the IDPROM should have the corresponding PMUnit name - used in the PlatformManager configuration as value. -- The Product Name should not be cryptic or in code words. The Product Name - should be obvious about the functionality of the PMUnit. Some examples - - Use SCM for a PMUnit containing CPU (not EAGLE or PLATFORM_NAME_SCM) - - Use SMB for a PMUnit containing the switch ASIC. - - Use PSU for Power Supply Unit - - Use FAN for FAN PmUnit, - - PDB, MCB, JUMPER, FCB etc., are other valid names -- Chassis EEPROM is not considered an IDPROM. - -### Chassis EEPROM - -- Chassis EEPROM will be modeled as just an ordinary EEPROM (within the holding - PMUnit). -- Chassis EEPROM should be a dedicated EEPROM. It MUST NOT also serve as a - FRU/PMUnit IDPROM described above. -- The Product Name field of the Chassis EEPROM must be the platform name. It - must be the same as what name is set above using dmidecode. - -### What is modeled as PMUnit in PlatformManager - -- Any unit which has an IDPROM (e.g., PSU, SCM, SMB). -- Any unit which does not have an IDPROM and can be field swapped and does not - have any devices like I2C, FPGA, CPLD, SPI, EEPROM etc. (e.g., FanTray). - -### What is not modeled as PMUnit in PlatformManager - -- Any unit which does not have an IDPROM and cannot be field swapped. -- If such unit has no devices, then it does not need modeling in - PlatformManager. -- If such unit has devices, then it will be modeled as part of another PMUnit - (referred to as hosting PMUnit). This is one case where one PMUnit could model - multiple FRUs together. In this case, when the device is re-spun, the IDPROM - of the hosting PMUnit MUST be updated to reflect the re-spin (e.g. product - version/sub-version number change). It should it also include IDPROM changes - of the hosting PMUnit. - -![drawing](/img/platform/platform_manager/platform_manager_flowchart.jpg) - -### Field-Replaceability - -- The unit must be a PMUnit for it to be field-replaceable. - -### Re-Spin - -- Any respin of a PMUnit, should involve update of the IDPROM of that PMUnit. -- Any respin of a non-PMUnit, should be bundled with changes in IDPROM of a - hosting PMUnit. -- The same platform config will be used for both re-spinned and original - platform. For the PMUnit which has difference across re-spin and original - platform, the platform config will have two definitions of the PMUnit - one - each for re-spin and original platform. Hardware Version will be used as a key - to look up the right PMUnit config. -- IDPROM content difference for the PMUnit across respins is a combination of - the below three keys in IDPROM. - - Product Production State - - Product Version - - Product Sub-Version -- TODO on Meta Side: Define PM config for re-spin. Flowchart: - -### PMUnit IDPROM I2C Addressing - -PMUnits of the same slot type should have the same address for the IDPROM. For -example, all PIM8DD and PIM16Q PMUnits, which are of slot type PIM, should have -the same address for their IDPROM. This enables these PMUnits to be plugged into -any PIM slot and be discovered by PlatformManager. - -### I2C Devices - -All I2C devices should be connected in one of the following ways: - -- Directly to an incoming I2C bus from a parent PMUnit -- Directly to a MUX or FPGA present within the PMUnit (or holding PMUnit) -- Directly to the CPU’s SMBus I2C Controller - -### PMUnit Presence - -PMUnit presence bits must be managed by the chips present in the parent PMUnit. - -### Root PmUnit - -This is the PmUnit which is used by Platform Manager as the starting node for -Platform exploration. The choice of root PmUnit should ensure all the above -conditions are met. diff --git a/docs/docs/platform/platform_manager/_category_.json b/docs/docs/platform/platform_manager/_category_.json new file mode 100644 index 0000000000000..cf2009c0bdc33 --- /dev/null +++ b/docs/docs/platform/platform_manager/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "PlatformManager", + "position": 1 +} diff --git a/docs/docs/platform/platform_manager/block_configs.md b/docs/docs/platform/platform_manager/block_configs.md new file mode 100644 index 0000000000000..d6de3e20e75c5 --- /dev/null +++ b/docs/docs/platform/platform_manager/block_configs.md @@ -0,0 +1,208 @@ +--- +id: block_configs +title: Block Configurations +sidebar_label: Block Configs +sidebar_position: 3 +--- + +# PlatformManager Block Configuration + +Block configurations allow defining multiple similar FPGA sub-devices using +offset calculation expressions. This reduces configuration verbosity when an +FPGA has many identical controllers at regular address offsets. + +## Overview + +Instead of listing each I2C adapter, LED controller, or transceiver controller +individually, block configs define a pattern with: +- A name prefix +- An offset calculation expression +- The number of instances + +## Offset Calculation Expressions + +Expressions support: +- Hexadecimal constants: `0x1000` +- Arithmetic: `+`, `-`, `*` +- Variables in braces (context-dependent, e.g., `{adapterIndex}`, `{busIndex}`) + +Available variables depend on the block config type: +- **I2C adapter blocks**: `{adapterIndex}`, `{startAdapterIndex}` +- **MDIO bus blocks**: `{busIndex}` +- **Port/LED configs**: `{portNum}`, `{startPort}`, `{ledNum}` + +The result is computed as a hexadecimal offset. + +### Example Calculation + +For `csrOffsetCalc: "0x1000 + ({adapterIndex} - {startAdapterIndex}) * 0x100"` +with `startAdapterIndex=1`: + +| adapterIndex | Calculation | Result | +|--------------|----------------------|--------| +| 1 | 0x1000 + (1-1)*0x100 | 0x1000 | +| 2 | 0x1000 + (2-1)*0x100 | 0x1100 | +| 3 | 0x1000 + (3-1)*0x100 | 0x1200 | + + +## I2C Adapter Block Config + +Creates multiple I2C adapters from an FPGA: + +```json +{ + "i2cAdapterBlockConfigs": [ + { + "pmUnitScopedNamePrefix": "SMB_I2C_ADAPTER", + "deviceName": "fbiob_i2c", + "csrOffsetCalc": "0x1000 + ({adapterIndex} - {startAdapterIndex}) * 0x100", + "iobufOffsetCalc": "0x2000 + ({adapterIndex} - {startAdapterIndex}) * 0x100", + "startAdapterIndex": 1, + "numAdapters": 8, + "numBusesPerAdapter": 1 + } + ] +} +``` + +This creates devices named `SMB_I2C_ADAPTER_1` through `SMB_I2C_ADAPTER_8`. + +| Field | Description | +|--------------------------|------------------------------------| +| `pmUnitScopedNamePrefix` | Prefix for generated device names | +| `deviceName` | Kernel driver name | +| `csrOffsetCalc` | Expression for CSR register offset | +| `iobufOffsetCalc` | Expression for I/O buffer offset | +| `startAdapterIndex` | Starting index for numbering | +| `numAdapters` | Number of adapters to create | +| `numBusesPerAdapter` | I2C buses per adapter (default: 1) | + +**Available variables:** + +| Variable | Description | +|-----------------------|-------------------------------------------------------------------------------------------| +| `{adapterIndex}` | Current adapter index (from `startAdapterIndex` to `startAdapterIndex + numAdapters - 1`) | +| `{startAdapterIndex}` | Value of `startAdapterIndex` field | + +## Transceiver Controller Block Config + +Creates multiple transceiver (QSFP/SFP) controllers: + +```json +{ + "xcvrCtrlBlockConfigs": [ + { + "pmUnitScopedNamePrefix": "XCVR_CTRL", + "deviceName": "fbiob_xcvr", + "csrOffsetCalc": "0x3000 + ({portNum} - {startPort}) * 0x4", + "iobufOffsetCalc": "0x4000 + ({portNum} - {startPort}) * 0x4", + "startPort": 1, + "numPorts": 32 + } + ] +} +``` + +This creates devices named `XCVR_CTRL_PORT_1` through `XCVR_CTRL_PORT_32`. + +| Field | Description | +|--------------------------|-------------------------------------| +| `pmUnitScopedNamePrefix` | Prefix for generated device names | +| `deviceName` | Kernel driver name | +| `csrOffsetCalc` | Expression for CSR register offset | +| `iobufOffsetCalc` | Expression for I/O buffer offset | +| `startPort` | Starting port number | +| `numPorts` | Number of ports | + +**Available variables:** + +| Variable | Description | +|---------------|----------------------------------------------------------------------| +| `{portNum}` | Current port number (from `startPort` to `startPort + numPorts - 1`) | +| `{startPort}` | Value of `startPort` field | + +## LED Controller Block Config + +Creates multiple LED controllers (typically per-port): + +```json +{ + "ledCtrlBlockConfigs": [ + { + "pmUnitScopedNamePrefix": "LED_CTRL", + "deviceName": "fbiob_led", + "csrOffsetCalc": "0x5000 + ({portNum} - {startPort}) * 0x8 + ({ledNum} - 1) * 0x4", + "iobufOffsetCalc": "0x6000 + ({portNum} - {startPort}) * 0x8 + ({ledNum} - 1) * 0x4", + "startPort": 1, + "numPorts": 32, + "ledPerPort": 2 + } + ] +} +``` + +This creates devices named `LED_CTRL_PORT_1_LED_1`, `LED_CTRL_PORT_1_LED_2`, +through `LED_CTRL_PORT_32_LED_2`. + +| Field | Description | +|--------------------------|----------------------------------- | +| `pmUnitScopedNamePrefix` | Prefix for generated device names | +| `deviceName` | Kernel driver name | +| `csrOffsetCalc` | Expression for CSR register offset | +| `iobufOffsetCalc` | Expression for I/O buffer offset | +| `startPort` | Starting port number | +| `numPorts` | Number of ports | +| `ledPerPort` | LEDs per port | + +**Available variables:** + +| Variable | Description | +|---------------|----------------------------------------------------------------------| +| `{portNum}` | Current port number (from `startPort` to `startPort + numPorts - 1`) | +| `{startPort}` | Value of `startPort` field | +| `{ledNum}` | Current LED number (from 1 to `ledPerPort`) | + +## MDIO Bus Block Config + +Creates multiple MDIO bus controllers: + +```json +{ + "mdioBusBlockConfigs": [ + { + "pmUnitScopedNamePrefix": "RTM_MDIO_BUS", + "deviceName": "fbiob_mdio", + "csrOffsetCalc": "0x200 + {busIndex} * 0x20", + "iobufOffsetCalc": "0x300 + {busIndex} * 0x4", + "numBuses": 4 + } + ] +} +``` + +This creates devices named `RTM_MDIO_BUS_0` through `RTM_MDIO_BUS_3`. + +| Field | Description | +|--------------------------|----------------------------------- | +| `pmUnitScopedNamePrefix` | Prefix for generated device names | +| `deviceName` | Kernel driver name | +| `csrOffsetCalc` | Expression for CSR register offset | +| `iobufOffsetCalc` | Expression for I/O buffer offset | +| `numBuses` | Number of MDIO buses | + +**Available variables:** + +| Variable | Description | +|--------------|----------------------------------------------| +| `{busIndex}` | Current bus index (from 0 to `numBuses - 1`) | + + +## Legacy Configs + +Block configs replace the deprecated individual configs: +- `i2cAdapterConfigs` -> `i2cAdapterBlockConfigs` +- `xcvrCtrlConfigs` -> `xcvrCtrlBlockConfigs` +- `ledCtrlConfigs` -> `ledCtrlBlockConfigs` +- `mdioBusConfigs` -> `mdioBusBlockConfigs` + +The legacy configs are still supported but should not be used for new platforms. diff --git a/docs/docs/platform/platform_manager/devmap.md b/docs/docs/platform/platform_manager/devmap.md new file mode 100644 index 0000000000000..4816c1034b266 --- /dev/null +++ b/docs/docs/platform/platform_manager/devmap.md @@ -0,0 +1,83 @@ +--- +id: devmap +title: Device Map (/run/devmap) +sidebar_label: Device Map +sidebar_position: 6 +--- + +# PlatformManager Device Map (/run/devmap) + +PlatformManager creates symbolic links under `/run/devmap/` to provide stable, +human-readable paths to platform devices. These symlinks should be used +instead of raw device paths like `/dev/i2c-5`. + +## Directory Structure + +``` +/run/devmap/ +├── i2c-busses/ +│ ├── OPTICS_1 -> /dev/i2c-5 +│ ├── OPTICS_2 -> /dev/i2c-6 +│ └── SMB_SENSOR_BUS -> /dev/i2c-12 +├── gpiochips/ +│ ├── SCM_CPLD_GPIO -> /sys/class/gpio/gpiochip432 +│ └── SMB_FPGA_GPIO -> /sys/class/gpio/gpiochip400 +├── sensors/ +│ ├── SCM_INLET_TEMP -> /sys/bus/i2c/devices/5-0048/hwmon/hwmon3 +│ └── SMB_OUTLET_TEMP -> /sys/bus/i2c/devices/12-0049/hwmon/hwmon5 +├── eeproms/ +│ ├── CHASSIS_EEPROM -> /sys/bus/i2c/devices/2-0050/eeprom +│ └── SCM_EEPROM -> /sys/bus/i2c/devices/5-0051/eeprom +├── watchdogs/ +│ └── FAN_WATCHDOG -> /dev/watchdog0 +├── fpgas/ +│ └── IOB_FPGA -> /sys/bus/pci/devices/0000:07:00.0 +├── cplds/ +│ └── SCM_CPLD -> /sys/bus/i2c/devices/5-0035 +├── spi-devices/ +│ └── BIOS_SPI -> /dev/spidev0.0 +├── mdio-busses/ +│ └── RTM_MDIO_0 -> /sys/class/mdio_bus/mdio-0 +├── xcvrs/ +│ └── XCVR_1 -> /sys/bus/pci/devices/.../xcvr_ctrl.1 +└── leds/ + └── PORT_1_LED_1 -> /sys/class/leds/port1:led1 +``` + +## Supported Device Groups + +| Group | Description | Example Target | +| ------------- | ---------------------------- | ---------------------------------------- | +| `i2c-busses` | I2C bus character devices | `/dev/i2c-N` | +| `gpiochips` | GPIO controller sysfs paths | `/sys/class/gpio/gpiochipN` | +| `sensors` | Hardware monitoring sensors | `/sys/bus/i2c/devices/.../hwmon/hwmonN` | +| `eeproms` | EEPROM sysfs paths | `/sys/bus/i2c/devices/.../eeprom` | +| `watchdogs` | Watchdog character devices | `/dev/watchdogN` | +| `fpgas` | FPGA PCI device sysfs paths | `/sys/bus/pci/devices/...` | +| `cplds` | CPLD I2C device sysfs paths | `/sys/bus/i2c/devices/...` | +| `spi-devices` | SPI character devices | `/dev/spidevN.M` | +| `mdio-busses` | MDIO bus sysfs paths | `/sys/class/mdio_bus/mdio-N` | +| `xcvrs` | Transceiver controller paths | Vendor-specific | +| `leds` | LED class device paths | `/sys/class/leds/...` | + +## Configuration + +Symlinks are configured in the platform configuration using +`symbolicLinkToDevicePath`: + +```json +{ + "symbolicLinkToDevicePath": { + "/run/devmap/eeproms/CHASSIS_EEPROM": "/[CHASSIS_EEPROM]", + "/run/devmap/sensors/SMB_INLET_SENSOR": "/SMB_SLOT@0/[INLET_SENSOR]", + "/run/devmap/sensors/FAN_CTRL": "/SMB_SLOT@0/[FAN_CTRL]" + } +} +``` + +## Naming Conventions + +Symlink names should: +- Use UPPER_CASE with underscores +- Be descriptive of the device function +- Include numeric suffixes for multiple instances (e.g., `OPTICS_1`, `OPTICS_2`) diff --git a/docs/docs/platform/platform_manager/exploration.md b/docs/docs/platform/platform_manager/exploration.md new file mode 100644 index 0000000000000..d90d4cfdd47bf --- /dev/null +++ b/docs/docs/platform/platform_manager/exploration.md @@ -0,0 +1,75 @@ +--- +id: exploration +title: Exploration Process +sidebar_label: Exploration +sidebar_position: 5 +--- + +# PlatformManager Exploration + +Platform exploration is the process of discovering and setting up all devices +on the platform based on the configuration file. + +## Exploration Workflow + +1. **Load Configuration**: Read platform-specific JSON config +2. **Package Management**: Install BSP RPM and load kernel modules +3. **Discover Root PmUnit**: Start from the root PmUnit (typically SCM) +4. **Recursive Exploration**: + - Read IDPROM to identify PmUnit type + - Create PCI sub-devices (from FPGA) + - Create I2C devices + - Create symbolic links in `/run/devmap/` + - Explore child slots recursively +5. **Publish Versions**: Read and publish firmware/hardware versions +6. **Report Status**: Log results and publish ODS counters + +## Exploration Status + +After exploration, the status is one of: + +| Status | Description | +| -------------------------------- | ------------------------------------------------------------ | +| `SUCCEEDED` | All devices explored without errors | +| `SUCCEEDED_WITH_EXPECTED_ERRORS` | Completed with expected errors (e.g., empty optional slots) | +| `FAILED` | Exploration encountered unexpected errors | +| `IN_PROGRESS` | Exploration is currently running | +| `UNSTARTED` | Exploration has not started | + +## Error Types + +Exploration errors are categorized by type: + +| Error Type | Description | +| ------------------------- | ----------------------------------------------------------------- | +| `I2C_DEVICE_CREATE` | Failed to create an I2C device | +| `I2C_DEVICE_REG_INIT` | Failed to initialize I2C device registers | +| `I2C_DEVICE_EXPLORE` | Failed during I2C device exploration | +| `RUN_DEVMAP_SYMLINK` | Failed to create symlink in /run/devmap | +| `PCI_DEVICE_EXPLORE` | Failed to explore PCI device | +| `PCI_SUB_DEVICE_CREATE_*` | Failed to create FPGA sub-device (I2C adapter, SPI, GPIO, etc.) | +| `IDPROM_READ` | Failed to read IDPROM | +| `SLOT_PM_UNIT_ABSENCE` | PmUnit not present in slot | +| `SLOT_PRESENCE_CHECK` | Failed to check slot presence | + + +## Querying Exploration Results + +### Via Thrift API + +Other FBOSS services can query exploration results programmatically using the +[Service API](/docs/platform/platform_manager/service_api): + +- `getLastPMStatus()` - Returns exploration status and failed devices map +- `getAllPmUnits()` - Returns all discovered PmUnits with their status + +### Via Logs + +Exploration results are logged at INFO level: + +``` +Successfully explored ... +``` + +Errors are logged at WARNING or ERROR level with device paths and error +messages. diff --git a/docs/docs/platform/platform_manager/modeling.md b/docs/docs/platform/platform_manager/modeling.md new file mode 100644 index 0000000000000..9b606edc64777 --- /dev/null +++ b/docs/docs/platform/platform_manager/modeling.md @@ -0,0 +1,138 @@ +--- +id: modeling +title: Modeling Requirements +sidebar_label: Modeling Requirements +sidebar_position: 2 +--- + +# Modeling Requirements + +This document describes the hardware modeling requirements for PlatformManager +configuration. + +## PMUnit + +Unit used for modeling in PlatformManager. It typically matches with a FRU, but +not always. PMUnit and FRU terminologies are used interchangeably in this +document. PMUnit and FRU differences will be explicitly called out when they are +not interchangeable. + +### What is modeled as PMUnit + +- Any unit which has an IDPROM (e.g., PSU, SCM, SMB). +- Any unit which does not have an IDPROM and can be field swapped and does not + have any devices like I2C, FPGA, CPLD, SPI, EEPROM etc. (e.g., FanTray). + +### What is not modeled as PMUnit + +- Any unit which does not have an IDPROM and cannot be field swapped. +- If such a unit has no devices (I2C, FPGA, CPLD, etc.), then it does not need + modeling in PlatformManager. +- If such a unit has devices (I2C, FPGA, CPLD, etc.), then it will be modeled as + part of another PMUnit (referred to as hosting PMUnit). This is one case where + one PMUnit could model multiple FRUs together. In this case, when the device + is re-spun, the IDPROM of the hosting PMUnit MUST be updated to reflect the + re-spin (e.g. product version/sub-version number change). It should also + include IDPROM changes of the hosting PMUnit. + +![PlatformManager Flowchart](/img/platform/platform_manager/platform_manager_flowchart.jpg) + +## EEPROM + +It should store content in [Meta EEPROM V6 format](/docs/platform/meta_eeprom_format_v6). + +## IDPROM + +- Any EEPROM which is used to identify the FRU/PMUnit type is called IDPROM. +- The IDPROM in the FRU MUST be connected in one of the following ways: + - Directly to an incoming I2C bus from the parent FRU. + - Directly to the CPU's SMBus I2C Controller. +- The Product Name field in the IDPROM should have the corresponding PMUnit name + used in the PlatformManager configuration as the value. +- The Product Name should not be cryptic or in code words. The Product Name + should be obvious about the functionality of the PMUnit. Some examples: + - Use SCM for a PMUnit containing CPU (not EAGLE or PLATFORM_NAME_SCM) + - Use SMB for a PMUnit containing the switch ASIC. + - Use PSU for Power Supply Unit + - Use FAN for FAN PmUnit + - PDB, MCB, JUMPER, FCB etc., are other valid names +- Chassis EEPROM is not considered an IDPROM. + +## Chassis EEPROM + +- Chassis EEPROM will be modeled as just an ordinary EEPROM (within the containing + PMUnit). +- Chassis EEPROM should be a dedicated EEPROM. It MUST NOT also serve as a + FRU/PMUnit IDPROM described above. +- The Product Name field of the Chassis EEPROM must be the platform name. It + must be the same as the output of `dmidecode -s system-product-name`. + +## Field-Replaceability + +- The unit must be a PMUnit (FRU) for it to be field-replaceable. + +## Re-Spin + +- Any respin of a PMUnit, should involve update of the IDPROM of that PMUnit. +- Any respin of a non-PMUnit, should be bundled with changes in IDPROM of a + hosting PMUnit. +- The same platform config will be used for both re-spun and original + platform. For the PMUnit which has difference across re-spin and original + platform, the platform config will have two definitions of the PMUnit - one + each for re-spin and original platform. Hardware Version will be used as a key + to look up the right PMUnit config. + +**Example configuration for re-spin:** + +```json +{ + "pmUnitConfigs": { + "SMB": [ + { + "productProductionState": 0, + "productVersion": 1, + "productSubVersion": 0, + "i2cDeviceConfigs": { ... } + }, + { + "productProductionState": 0, + "productVersion": 2, + "productSubVersion": 0, + "i2cDeviceConfigs": { ... } + } + ] + } +} +``` + +PlatformManager reads the IDPROM to determine which PMUnit config to use. +- IDPROM content difference for the PMUnit across respins is a combination of + the below three keys in IDPROM: + - Product Production State + - Product Version + - Product Sub-Version + +## PMUnit IDPROM I2C Addressing + +PMUnits of the same slot type should have the same address for the IDPROM. For +example, all PIM8DD and PIM16Q PMUnits, which are of slot type PIM, should have +the same address for their IDPROM. This enables these PMUnits to be plugged into +any PIM slot and be discovered by PlatformManager. + +## I2C Devices + +All I2C devices should be connected in one of the following ways: + +- Directly to an incoming I2C bus from a parent PMUnit +- Directly to a MUX or FPGA present within the PMUnit (or containing PMUnit) +- Directly to the CPU's SMBus I2C Controller + +## PMUnit Presence + +PMUnit presence bits must be managed by the chips present in the parent PMUnit. + +## Root PmUnit + +This is the PmUnit which is used by Platform Manager as the starting node for +Platform exploration. The choice of root PmUnit should ensure all the above +conditions are met. diff --git a/docs/docs/platform/platform_manager/overview.md b/docs/docs/platform/platform_manager/overview.md new file mode 100644 index 0000000000000..7468a7e105e7d --- /dev/null +++ b/docs/docs/platform/platform_manager/overview.md @@ -0,0 +1,91 @@ +--- +id: overview +title: PlatformManager Overview +sidebar_label: Overview +sidebar_position: 1 +slug: /platform/platform_manager +--- + +# PlatformManager + +## Introduction + +The objective of PlatformManager is to generalize low level device setup and +management across platforms. This avoids platform specific customized software +logic. All the Platform specific information should be encoded in configuration +files such that the same generic software can run on all platforms. + +## Functions of Interest + +- **Inventory Management**: A switch system could have different FRUs. + PlatformManager is responsible for building the inventory of the FRUs in a + switch. PlatformManager needs to monitor in real time what FRU has been + removed or inserted into the switch. It provides an interface to software + components within and outside of PlatformManager about the FRUs and the FRU + EEPROM contents. +- **Dynamic Device Management**: When an FRU has been detected by the Inventory + Management, Dynamic Device Management will load the necessary kmods and enable + communication with the FRU. PlatformManager will also ensure that the FRU + devices get consistent naming within each platform irrespective of how it is + plugged in and across reboots. + +## Components + +- **Platform Specific Configuration (provided by Vendors):** It has information + about the FRUs, I2C devices and how they can be accessed. The configuration + format is defined in [platform_manager_config.thrift](https://github.com/facebook/fboss/blob/main/fboss/platform/platform_manager/platform_manager_config.thrift). See here for a [sample](https://github.com/facebook/fboss/blob/main/fboss/platform/configs/sample/platform_manager.json) config. +- **Platform Agnostic Software (developed by Meta):** Meta will write and + maintain the software. The [source code is here](https://github.com/facebook/fboss/tree/main/fboss/platform/platform_manager). + +## Documentation + +| Topic | Description | +| ---------------------------------------------------------------------- | ------------------------------------------------ | +| [Modeling Requirements](/docs/platform/platform_manager/modeling) | PMUnit, IDPROM, and hardware modeling rules | +| [Block Configurations](/docs/platform/platform_manager/block_configs) | Bulk FPGA sub-device configuration | +| [Presence Detection](/docs/platform/platform_manager/presence_detection) | GPIO and sysfs-based FRU presence detection | +| [Exploration Process](/docs/platform/platform_manager/exploration) | How platform discovery works, status and errors | +| [Device Map](/docs/platform/platform_manager/devmap) | Symbolic link structure for stable device paths | +| [Running PlatformManager](/docs/platform/platform_manager/running) | Command-line flags, run modes, systemd integration | +| [Package Manager](/docs/platform/platform_manager/package_manager) | BSP RPM management and kmods loading | +| [Thrift Service API](/docs/platform/platform_manager/service_api) | Service endpoints for querying platform state | + +## Workflow + +### On Initialization + +- Determine Platform Type using dmidecode set by BIOS. For details, refer to + the [Meta Switch Architecture](/docs/architecture/meta_switch_architecture) + documentation, which also links to the BMC-Lite specification. +- Load the platform specific PlatformManager Config +- Discover the devices (by polling present registers or reading sysfs paths) + based on the config and load the necessary kmods. + +### Periodically + +- Detect devices by listening to present polling/interrupts (by polling for now, + since interrupts are not full-fledged) +- On device detection, use the PlatformManager Config to set up the device and + load the necessary kmods. + +## Usage + +- PlatformManager will be used at provisioning/boot-up and be executed before + any FRU/EEPROM/FW access. +- PlatformManager will be run in steady state of the switch to facilitate + dynamic detection of devices. + +## Vendor Expectations + +- **Configuration**: Vendor will provide the platform specific configuration + according to the [specification](https://github.com/facebook/fboss/blob/main/fboss/platform/platform_manager/platform_manager_config.thrift) + in fboss repository. +- **Handoff**: Vendor will run PlatformManager as part of the handoff. The running + and testing instructions will be provided in fboss repository. +- **PCI Device IDs**: All PCI devices (e.g., FPGA) should have unique Vendor ID and + Device ID in the PCI configuration space. +- **Dmidecode Setup**: The platform type should be obtainable through dmidecode. + See [Meta Switch Architecture](/docs/architecture/meta_switch_architecture) for + details. +- **BSP**: The BSP should follow the [BSP requirements specification](/docs/platform/bsp_development_requirements) + to enable device discovery and setup. diff --git a/docs/docs/platform/platform_manager/package_manager.md b/docs/docs/platform/platform_manager/package_manager.md new file mode 100644 index 0000000000000..f79443b7622b5 --- /dev/null +++ b/docs/docs/platform/platform_manager/package_manager.md @@ -0,0 +1,88 @@ +--- +id: package_manager +title: Package Manager +sidebar_label: Package Manager +sidebar_position: 8 +--- + +# PlatformManager Package Manager + +The Package Manager component of PlatformManager handles BSP (Board Support +Package) RPM management and kernel module loading. + +## Overview + +Before platform exploration can occur, the correct kernel modules must be +loaded. The Package Manager: + +1. Installs the BSP RPM if not already installed +2. Unloads any existing BSP kernel modules +3. Loads required kernel modules in the correct order +4. Handles watchdog devices during kmod operations + +## BSP RPM Configuration + +The platform configuration specifies the BSP RPM: + +```json +{ + "bspKmodsRpmName": "vendor_bsp_kmods", + "bspKmodsRpmVersion": "1.0.0", + "requiredKmodsToLoad": ["module1", "module2"] +} +``` + +- `bspKmodsRpmName`: Base name of the RPM package +- `bspKmodsRpmVersion`: Required version +- `requiredKmodsToLoad`: Modules to load before exploration (most modules are + loaded automatically during device creation) + +## Kernel Modules Management + +BSP vendors provide a `kmods.json` file as part of BSP RPM that lists the kernel modules. This file is located at `/usr/local/{vendor}_bsp/kmods.json` and is used by PlatformManager to unload the kernel modules. + +```json +{ + "bspKmods": [ + "vendor_fpga_driver", + "vendor_i2c_driver", + "vendor_fan_driver" + ], + "sharedKmods": [ + "i2c_mux_pca954x", + "at24" + ] +} +``` + +When `--reload_kmods=true` or when the BSP version changes: + +1. Close any open watchdog devices (prevents system reset) +2. Unload `bspKmods` in reverse order +3. Unload `sharedKmods` in reverse order +4. Run `depmod` to refresh module dependencies +5. Load `requiredKmodsToLoad` modules +6. Proceed with platform exploration (which loads modules on-demand) + +## Local RPM Testing + +For development, a local RPM can be tested by specifying its path: + +```bash +platform_manager --local_rpm_path=/path/to/_bsp_kmods-.rpm +``` + +This bypasses normal RPM resolution and installs the specified file directly. + +Note: The actual BSP RPM name usually contains version information like `_bsp_kmods-.x86_64--1`. For example, `fboss_bsp_kmods-6.11.1-1.fboss.el9.x86_64-4.0.0-1`. + + +## Disabling Package Management + +Package management can be skipped for debugging purposes: + +```bash +platform_manager --enable_pkg_mgmnt=false +``` + +This assumes all required modules are already loaded. diff --git a/docs/docs/platform/platform_manager/presence_detection.md b/docs/docs/platform/platform_manager/presence_detection.md new file mode 100644 index 0000000000000..42186e3730848 --- /dev/null +++ b/docs/docs/platform/platform_manager/presence_detection.md @@ -0,0 +1,129 @@ +--- +id: presence_detection +title: Presence Detection +sidebar_label: Presence Detection +sidebar_position: 4 +--- + +# PlatformManager Presence Detection + +PlatformManager supports detecting whether a PmUnit (FRU) is physically present +in a slot. This enables hot-plug detection and proper handling of optional +components. + +## Overview + +Presence detection determines if a PmUnit is plugged into a slot before +attempting to explore it. This prevents errors when accessing empty slots and +enables reporting of which slots are populated. + +## Detection Methods + +### GPIO Line Handle + +Uses a GPIO line from a gpio chip to detect presence: + +```json +{ + "presenceDetection": { + "gpioLineHandle": { + "devicePath": "/[SCM_CPLD_GPIO]", + "lineIndex": 5, + "desiredValue": 1 + } + } +} +``` + +- `devicePath`: DevicePath to the gpio chip device +- `lineIndex`: GPIO line number (0-indexed) +- `desiredValue`: Value indicating presence (typically 0 or 1) + +### Sysfs File Handle + +Uses a sysfs file to detect presence: + +```json +{ + "presenceDetection": { + "sysfsFileHandle": { + "devicePath": "/[SMB_CPLD]", + "presenceFileName": "pim1_present", + "desiredValue": 1 + } + } +} +``` + +- `devicePath`: DevicePath to the device exposing the sysfs file +- `presenceFileName`: Name of the sysfs file containing presence status +- `desiredValue`: Value indicating presence + +## Configuration + +Presence detection is configured per-slot in `SlotConfig`: + +```json +{ + "outgoingSlotConfigs": { + "PIM_SLOT@0": { + "slotType": "PIM_SLOT", + "presenceDetection": { + "gpioLineHandle": { + "devicePath": "/[IOB_GPIO]", + "lineIndex": 0, + "desiredValue": 1 + } + }, + "outgoingI2cBusNames": ["IOB_I2C_0@0"] + } + } +} +``` + +## Presence Info Response + +When querying PmUnit info via the Thrift API, presence information is included: + +``` +PmUnitInfo { + name: "PIM8DD", + presenceInfo: { + presenceDetection: { ... }, + actualValue: 1, + isPresent: true + }, + successfullyExplored: true +} +``` + +- `actualValue`: The raw value read from the GPIO or sysfs file +- `isPresent`: Whether the actual value matches the desired value + +## Exploration Behavior + +1. If `presenceDetection` field is configured for a slot: + - PlatformManager checks presence before exploring + - If FRU is absent: the slot is skipped and a `SLOT_PM_UNIT_ABSENCE` error is + recorded + - If FRU is present: normal exploration proceeds + +2. If `presenceDetection` field is NOT configured: + - The slot is assumed to always be populated + - Exploration proceeds directly + +## Expected Empty Slots + +Some slots may be expected to be empty. These are tracked separately from +unexpected failures. When a slot is expected to be empty, its absence is +reported as `SUCCEEDED_WITH_EXPECTED_ERRORS` rather than `FAILED`. Currently, only `PSU_SLOT` types are treated as expected-to-be-empty. + +## Hardware Requirements + +For GPIO-based presence detection: +- The GPIO chip must be created before checking presence +- Parent PmUnit must be explored first + +For sysfs-based presence detection: +- The device exposing the sysfs file must be created first +- The sysfs file must be readable diff --git a/docs/docs/platform/platform_manager/running.md b/docs/docs/platform/platform_manager/running.md new file mode 100644 index 0000000000000..3b547d8407217 --- /dev/null +++ b/docs/docs/platform/platform_manager/running.md @@ -0,0 +1,55 @@ +--- +id: running +title: Running PlatformManager +sidebar_label: Running +sidebar_position: 7 +--- + +# Running PlatformManager + +PlatformManager can run in two modes: one-shot mode for initial platform setup, +or daemon mode for continuous operation with a Thrift service. + +## Command-Line Flags + +### Core Flags + +| Flag | Default | Description | +| ---------------------- | ------- | ---------------------------------------------------------------------------------------- | +| `--run_once` | `true` | If true, explore platform once and exit. If false, run as a daemon with Thrift service. | +| `--thrift_port` | `5975` | Port for the Thrift service (only used when `--run_once=false`) | +| `--explore_interval_s` | `60` | Interval between exploration cycles in daemon mode | + +### Package Management Flags + +| Flag | Default | Description | +| -------------------- | ------- | ---------------------------------------------------------------------- | +| `--enable_pkg_mgmnt` | `true` | Enable BSP package management (RPM download and installation) | +| `--reload_kmods` | `false` | Force reload of kernel modules even if already loaded | +| `--local_rpm_path` | `""` | Path to a local RPM file for testing (bypasses normal RPM resolution) | + +## Running Modes + +### One-Shot Mode (Default) + +```bash +platform_manager --run_once=true +``` + +This mode: +1. Loads the platform configuration +2. Processes BSP packages (installs RPMs, loads kmods) +3. Explores the platform and creates device symlinks +4. Exits with status 0 on success, 1 on failure + + +### Daemon Mode + +```bash +platform_manager --run_once=false +``` + +Performs the same steps as one-shot mode, but instead of exiting: +1. Starts the Thrift service on the configured port +2. Continues running to handle API requests +3. Sends `sd_notify` ready signal for systemd integration diff --git a/docs/docs/platform/platform_manager/service_api.md b/docs/docs/platform/platform_manager/service_api.md new file mode 100644 index 0000000000000..84d376631b429 --- /dev/null +++ b/docs/docs/platform/platform_manager/service_api.md @@ -0,0 +1,85 @@ +--- +id: service_api +title: Thrift Service API +sidebar_label: Service API +sidebar_position: 9 +--- + +# PlatformManager Service API + +PlatformManager exposes a Thrift service on port 5975 (default) that allows +querying platform state and exploration status. + +## Service Endpoints + +The Thrift interface is defined in +[platform_manager_service.thrift](https://github.com/facebook/fboss/blob/main/fboss/platform/platform_manager/platform_manager_service.thrift). + +### getPlatformName() + +Returns the platform name as detected from dmidecode. + +**Response**: `string` + +### getAllPmUnits() + +Returns information about all discovered PmUnits in the platform. + +**Response**: `PmUnitsResponse` +- `pmUnits`: Map from slot path to `PmUnitInfo` + +### getLastPMStatus() + +Returns the status of the most recent platform exploration. + +**Response**: `PlatformManagerStatus` +- `explorationStatus`: One of: + - `SUCCEEDED`: All devices explored successfully + - `IN_PROGRESS`: Exploration currently running + - `FAILED`: Exploration encountered errors + - `SUCCEEDED_WITH_EXPECTED_ERRORS`: Completed with expected errors (e.g., empty slots) + - `UNSTARTED`: Exploration has not started +- `lastExplorationTime`: Unix timestamp of last exploration completion +- `failedDevices`: Map of device paths to their exploration errors + +### getPmUnitInfo(slotPath) + +Returns information about a specific PmUnit at the given slot path. + + +**Response**: `PmUnitInfoResponse` +- `pmUnitInfo`: Contains name, version, presence info, and exploration status + + +### getBspVersion() + +Returns BSP package version information. + +**Response**: `BspVersionResponse` +- `bspBaseName`: Base name of the BSP RPM +- `bspVersion`: Version string of the installed BSP +- `kernelVersion`: Running kernel version + + +### getEepromContents(slotPath) + +Returns the EEPROM contents for the PmUnit at the given slot path. + + +## Client Usage + +The PlatformManager service is consumed programmatically by other FBOSS services. +For example, `sensor_service` uses `PmClientFactory` to connect. The following is an example of using a custom thrift client to query the service. + +``` +./pm_thrift_client getPlatformName +MONTBLANC + + +./pm_thrift_client getBspVersion +{ + "bspBaseName": "fboss_bsp_kmods", + "bspVersion": "3.4.0-1", + "kernelVersion": "6.12.49-1.el9.x86_64" +} +```