Further docs

docs/how-to/new_raspi.md

@@ -4,7 +4,7 @@
 
 IMPORTANT: normal operation for setting up a new Raspberry Pi is to use the provided pre-built image sdcards. See [Commissioning a new Raspberry Pi Server](../tutorials/commissioning_raspi.md).
 
-This page is only useful if you need to create a completely new image, perhaps because a new version of Raspberry Pi OS has been released.
+This page is only useful if you need to create a completely new sdcard with a new image, perhaps because a new version of Raspberry Pi OS has been released.
 
 If you just want to update an existing image, e.g. to update the version of usb-remote it uses see [Updating an Existing Raspberry Pi Boot Image](updating_raspi_image.md).
 
@@ -19,28 +19,24 @@ You will need:
 
 ## Step 1 Image the microSD Card with Raspberry Pi OS
 
-1. Download the latest Raspberry Pi OS Full image from the [Raspberry Pi website](https://www.raspberrypi.com/software/operating-systems/).
-    - Note you only need full version if you plan to use the Raspberry Pi's GUI capabilities. Otherwise the "Raspberry Pi OS Lite" version is sufficient. Use full if you want to connect to the Government WiFi network which requires a GUI for setup.
+1. Download the latest Raspberry Pi OS Lite image from the [Raspberry Pi website](https://www.raspberrypi.com/software/operating-systems/).
+    - If you plan to use the Raspberry Pi's GUI capabilities, use "Raspberry Pi OS Full" version. Use full if you want to connect to the GovWiFi network which requires a GUI for setup.
 1. Use `lsblk` to get a list of block devices on your system before inserting the microSD card.
 1. Insert the microSD card into your card reader and connect it to your computer.
-1. Use `lsblk` again to identify the device name of the microSD card (e.g. `/dev/sdX`).
+1. Use `lsblk` again to identify the device name of the microSD card (e.g. `/dev/sdb`).
 1. uncompress the downloaded Raspberry Pi OS image.
     ```bash
      cd ~/Downloads
-     unxz ./2025-12-04-raspios-trixie-arm64-full.img.xz
+     unxz ./2025-12-04-raspios-trixie-arm64.img.xz
      ```
-1. Use `dd` to write the Raspberry Pi OS image to the microSD card. Replace `/dev/sdX` with the actual device name of your microSD card.
+1. Use `dd` to write the Raspberry Pi OS image to the microSD card. Replace `/dev/sdX` with the actual device name of your microSD card. Be very careful with this command as it will overwrite the specified device.
     ```bash
-    sudo dd if=./2025-12-04-raspios-trixie-arm64-full.img of=/dev/sdX bs=4M status=progress conv=fsync
-    ```
-1. Once the `dd` command has completed, run `sync` to ensure all data has been written to the microSD card.
-    ```bash
-    sync
+    sudo dd if=./2025-12-04-raspios-trixie-arm64.img of=/dev/sdX bs=4M status=progress conv=fsync
     ```
 
 ## Step 2 Configure the Raspberry Pi OS Image
 
-1. Mount the microSD card boot partition.
+1. Mount the microSD card boot partition. Replace `/dev/sdX1` with the actual device name of the boot partition of your microSD card.
     ```bash
     mkdir sdcard-bootfs
     sudo mount /dev/sdX1 sdcard-bootfs
@@ -56,14 +52,15 @@ You will need:
     ```
 1. If you need a static IP address for wired ethernet, edit `firmware/cmdline.txt`.
     ```bash
-    sudo vim firmware/cmdline.txt
-    # or sudo vim cmdline.txt if `firmware` directory does not exist
-    # add <space>ip=<your_static_ip_address> at the end of the single line in the file, replacing with your desired static IP address
+    sudo vim /boot/cmdline.txt
+    # add " ip=<your_static_ip_address>" at the end of the single line in the file.
     ```
 1. Finally unmount the boot partition.
     ```bash
     cd ..
     sudo umount sdcard-bootfs
+    # there may also be a second mount point:
+    sudo umount /dev/sdX
     rmdir sdcard-bootfs
     ```
 1. NOTE: the static IP configuration above must be undone before making a production image to be used on multiple Raspberry Pis. Instead we want the image to use DHCP only and be isolated from the internet.
@@ -85,6 +82,11 @@ You will need:
     sudo apt update
     sudo apt upgrade -y
     ```
+1. Take this opportunity to write down the Raspberry Pi's MAC address for future reference.
+    ```bash
+    ip link show eth0
+    # look for "link/ether xx:xx:xx:xx:xx:xx"
+    ```
 
 ## Step 4 Install and Configure usbip
 1. Add the kernel modules to `/etc/modules` so they load at boot.
@@ -129,3 +131,98 @@ You will need:
     sudo systemctl enable --now usb-remote
     sudo systemctl status usb-remote
     ```
+
+## Step 6 image-backup
+
+1. Clone the image-backup repository.
+    ```bash
+    cd ~
+    git clone https://github.com/seamusdemora/RonR-RPi-image-utils.git
+    ```
+1. Install image-backup.
+    ```bash
+    sudo install --mode=755 ~/RonR-RPi-image-utils/image-* /usr/local/sbin
+    ```
+
+## Step 7 Clean Up The Settings for Production
+
+The master sdcard image wants to use DHCP only and be isolated from the internet so we need to undo any temporary configuration changes made earlier. This allows us to re-use the image on multiple Raspberry Pis which will each get their own IP address via DHCP.
+
+1. Disable the static IP address if you set one up.
+    ```bash
+    sudo apt install vim
+    sudo vim /boot/firmware/cmdline.txt
+    # remove " ip=<your_static_ip_address>" that you added earlier
+    ```
+1. Disable Wifi if you enabled it.
+    ```bash
+    sudo vim /boot/firmware/config.txt
+    # add the following lines at the end of the file
+    dtoverlay=disable-wifi
+    dtoverlay=disable-bt
+    ```
+
+## Step 8 Create a Backup Image of the microSD Card
+
+Before backing up the image we put the SD card into read-only mode. This avoids wearing out the SD card and makes the Pi reset to a clean state on each boot.
+
+1. Set the SD card to read-only mode with an overlay filesystem in RAM.
+    ```bash
+    sudo raspi-config nonint do_overlayfs 0
+    # DO NOT reboot until the backup image has been created
+    # ALSO: ignore warnings about fstab has been modifiedsyn
+    ```
+
+1. Insert a USB stick into the Raspberry Pi to store the backup image.
+
+1. use `lsblk` to identify the device name of the USB stick 1st partition (e.g. `/dev/sda1`). It should already be mounted under `/media/local/xxxx`. If there is no mount then create a mount point and mount it manually:
+    ```bash
+    sudo mkdir -p /media/local/usb
+    sudo mount /dev/sda1 /media/local/usb
+    ```
+
+1. Run the image-backup script to create a backup image of the microSD card to the USB stick. Replace `/media/local/xxxx` with the actual mount point of your USB stick and adjust the output file path as needed.
+    ```bash
+    sudo image-backup
+    # when promted for output file, use something like:
+    /media/local/usb/raspi-usb-remote-2025.12.20.img
+    # choose the defaults for the other prompts and y to confirm file creation.
+    ```
+1. sync and unmount the USB stick.
+    ```bash
+    sync
+    sudo umount /media/local/xxxx
+    ```
+1. You can now power off the Raspberry Pi and remove the USB stick.
+    ```bash
+    sudo poweroff
+    ```
+
+## Step 9 Prepare the Backup Image for Distribution
+
+The image file you have created has removed the empty space in the root partition to make it nice and small. The contents of boot partition must be updated to tell the Raspberry Pi to expand the root partition to fill the microSD card on first boot. The `pi-shrink` script does this for you.
+
+1. Get the `pi-shrink` script.
+    ```bash
+    curl -fsSO https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
+    chmod +x pishrink.sh
+    ```
+1. Run `pi-shrink` on the backup image you created in Step 8.
+    ```bash
+    sudo ./pishrink.sh /path/to/your/raspi-usb-remote-2025.12.20.img
+    ```
+
+You can ignore the warning "Filesystem already shrunk to smallest size. Skipping filesystem shrinking".
+
+That's it. Your img file can now be used to create additional Raspberry Pi usb-remote servers as needed.
+
+
+## Conclusion
+
+You now have a backup image of your configured Raspberry Pi microSD card that you can use to create additional Raspberry Pi servers as needed. To restore the image to another microSD card, use the `dd` command as described in Step 1, replacing the input file with your backup image file.
+
+The Raspberry Pi you have just been working with is now ready to be used as a production server. Simply power it back on and it will boot into read-only mode with usb-remote installed and ready to use.
+
+You should configure your router or `infoblox` to assign it a DHCP reservation so it always gets the same IP address. To do this requires knowing the Raspberry Pi's MAC address.
+
+See [Commissioning Additional Raspberry Pi Servers](../tutorials/commissioning_raspi.md) for instructions on deploying the backup image to new Raspberry Pis.

docs/tutorials/commissioning_raspi.md

@@ -1,3 +1,60 @@
-# Commissioning a Raspberry new Raspberry Pi Server
+# Commissioning a new Raspberry Pi as a usb-remote Server
 
-TODO: Write this page.
+## Introduction
+
+To add remote USB device support requires a usb-remote server at the location of the USB devices. e.g. In a beamline's experimental hutch. This tutorial describes how to commission a new Raspberry Pi to act as a usb-remote server.
+
+
+## Step 1: Obtain and Assemble Recommended Hardware
+
+See [Recommended Hardware](../reference/recommended_hardware.md) for the list of recommended hardware to use for a Raspberry Pi usb-remote server.
+
+Any Raspberry Pi 4 or 5 with at least 4GB RAM and at least 16GB microSD card is suitable.
+
+TODO: some notes on assembly go here.
+
+## Step 2a: Obtain Raspberry Pi usb-remote Server microSD Card
+
+If you have a supply of pre-built Raspberry Pi usb-remote server sdcard images you can skip this step and go to `Step 3: Extract the Raspberry Pi MAC Address`.
+
+At DLS these can be obtained from TODO: where?
+
+## Step 2b: Flash the Raspberry Pi usb-remote Server Image
+
+Alternatively, flash your own card.
+
+1. Obtain the latest Raspberry Pi usb-remote server image.
+    - At DLS this is available at /dls_sw/apps TODO: add path here.
+    - giles' latest image can be downloaded from here: https://drive.google.com/file/d/1pvCIkpnDC90Z086w6k2WjTKtdAcx9RPU/view?usp=sharing
+    - Alternatively, build your own image using the instructions at [Create a New Raspberry Pi Boot Image From Scratch](../how-to/new_raspi.md).
+1. Insert a microSD card of at least 16GB capacity into a card reader connected to your computer.
+1. Use `lsblk` to identify the device name of the microSD card (e.g. `/dev/sdb`).
+1. Flash the image to a microSD card. **CAREFUL** - replace `/dev/sdX` with the correct device name for your microSD card and remember that this will overwrite the specified device.
+    ```bash
+    sudo dd if=./raspberry_pi_usb-remote_server.img of=/dev/sdX bs=4M status=progress conv=fsync
+    ```
+
+## Step 3: Extract the Raspberry Pi MAC Address
+
+- TODO: this will involve booting the Pi with a printer plugged in.
+- Alternatively, if you have access to a monitor and keyboard you can boot the Pi and run `ip link show eth0` to get the MAC address of the `eth0` interface.
+
+
+## Step 4: Configure an IP Address for the Raspberry Pi
+
+- TODO: infoblox instructions go here.
+
+## Step 5: Connect the Raspberry Pi to the Network and Power it On
+
+- Connect the Raspberry Pi to the network using a wired ethernet connection.
+- Power on the Raspberry Pi using the USB-C power supply.
+
+## Step 6: Verify the New Server is Visible to the usb-remote Client
+On any linux machine that can route to the new Raspberry Pi server IP, run:
+
+```bash
+uvx usb-remote config add-server <raspberry_pi_ip_address>
+uvx usb-remote list
+```
+
+You should see the new server listed without errors.

src/usb_remote/client.py

@@ -63,7 +63,7 @@ def send_request(
             sock.sendall(request.model_dump_json().encode("utf-8"))
 
             response = sock.recv(4096).decode("utf-8")
-            logger.debug("Received response from server")
+            logger.debug(f"Received response from server: {response}")
             # Parse response using TypeAdapter to handle union types
             response_adapter = TypeAdapter(
                 ListResponse | DeviceResponse | ErrorResponse
@@ -119,6 +119,8 @@ def list_devices(
             assert isinstance(response, ListResponse)
             results[server] = response.data
             logger.debug(f"Server {server}: {len(response.data)} devices")
+        except DeviceNotFoundError:
+            pass  # expect not to find the device on all servers
         except Exception as e:
             logger.warning(f"Failed to query server {server}: {e}")
             results[server] = []