Skip to content

feat: add config file support to configure docker images #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
KatyaRyazantseva wants to merge 5 commits into
base: main
Choose a base branch
from
+589 −48
Draft

feat: add config file support to configure docker images #92

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f0f4276
feat: add config file support to configure docker images
KatyaRyazantseva Jan 16, 2026
698568d
Merge branch 'main' into add-config-file
KatyaRyazantseva Jan 19, 2026
39dc7a6
fix: move image to the validator-config.yml
KatyaRyazantseva Jan 21, 2026
b4c1b32
Merge remote-tracking branch 'upstream/main' into add-config-file
KatyaRyazantseva Jan 26, 2026
c347b73
Merge branch 'main' into add-config-file
KatyaRyazantseva Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ data
*.old
.DS_Store

# User configuration files
user-config.yml

# Generated genesis files (created by generate-genesis.sh)
config.yaml
validators.yaml
Expand Down
55 changes: 52 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,44 @@ NETWORK_DIR=local-devnet ./spin-node.sh --node zeam_0 --generateGenesis --popupT
NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --metrics
```

### Using custom Docker images

You can override default Docker images using the `--config-file` flag. This is useful for testing custom builds or using specific versions without modifying the codebase.

**Basic usage (without custom images):**
```sh
# Uses default images from validator-config.yaml in your network directory
NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis
```

**With custom config file:**
```sh
# Override specific node images
NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --config-file user-config.yml
```

**Example config file (user-config.yml):**
```yaml
nodes:
- name: zeam_0
image: blockblaz/zeam:feature-branch
- name: ream_0
image: ghcr.io/reamlabs/ream:v2.0
```

**Testing a specific client build:**
```sh
# Create custom config file for zeam <your-PATH>/my-zeam-config.yml
nodes:
- name: zeam_0
image: blockblaz/zeam:custom-tag

# Run with custom zeam image
NETWORK_DIR=local-devnet ./spin-node.sh --node zeam_0 --config-file <your-PATH>/my-zeam-config.yml
```

Only specify nodes you want to override - others will use their defaults from `validator-config.yaml`.

## Args

1. `NETWORK_DIR` is an env to specify the network directory. Should have a `genesis` directory with genesis config. A `data` folder will be created inside this `NETWORK_DIR` if not already there.
Expand Down Expand Up @@ -119,7 +157,12 @@ NETWORK_DIR=local-devnet ./spin-node.sh --node all --generateGenesis --metrics
- If not provided, defaults to `latest` for zeam, ream, and lantern, and `dd67521` for qlean
- The script will automatically pull the specified Docker images before running containers
- Example: `--tag devnet0` or `--tag devnet1`
11. `--metrics` enables metrics collection on all nodes. When specified, each client will activate its metrics endpoint according to its implementation. Metrics ports are configured per node in `validator-config.yaml`.
11. `--config-file` specifies a custom configuration file to override default Docker images for specific nodes.
- Path to a YAML file containing node image overrides (e.g., `user-config.yml` or `/path/to/my-config.yml`)
- Only nodes specified in the config file are overridden; others use defaults from `validator-config.yaml`
- See [Using custom Docker images](#using-custom-docker-images) scenario for usage examples
- Example: `--config-file user-config.yml` or `--config-file /path/to/custom-config.yml`
12. `--metrics` enables metrics collection on all nodes. When specified, each client will activate its metrics endpoint according to its implementation. Metrics ports are configured per node in `validator-config.yaml`.

### Clients supported

Expand Down Expand Up @@ -150,12 +193,18 @@ The quickstart uses separate directories for local and Ansible deployments:

```
lean-quickstart/
├── local-devnet/ # Local development
├── client-cmds/ # Client command scripts
│ ├── zeam-cmd.sh
│ ├── ream-cmd.sh
│ └── ...
├── user-config.yml.example # Example custom config (copy to user-config.yml)
├── user-config.yml # Your custom image overrides (gitignored)
├── local-devnet/ # Local development
│ ├── genesis/
│ │ └── validator-config.yaml # Local IPs (127.0.0.1)
│ └── data/ # Node data directories
│
└── ansible-devnet/ # Ansible/remote deployment
└── ansible-devnet/ # Ansible/remote deployment
├── genesis/
│ └── validator-config.yaml # Remote IPs (your server IPs)
└── data/ # Node data directories
Expand Down
4 changes: 4 additions & 0 deletions ansible-devnet/genesis/validator-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ validators:
- name: "zeam_0"
# node id 7d0904dc6d8d7130e0e68d5d3175d0c3cf470f8725f67bd8320882f5b9753cc0
# peer id 16Uiu2HAkvi2sxT75Bpq1c7yV2FjnSQJJ432d6jeshbmfdJss1i6f
image: blockblaz/zeam:latest
privkey: "bdf953adc161873ba026330c56450453f582e3c4ee6cb713644794bcfdd85fe5"
enrFields:
# verify /ip4/127.0.0.1/udp/9000/quic-v1/p2p/16Uiu2HAkvi2sxT75Bpq1c7yV2FjnSQJJ432d6jeshbmfdJss1i6f
Expand All @@ -19,6 +20,7 @@ validators:
- name: "ream_0"
# node id bc531fc1a99a896acb45603f28a32f81ae607480af46435009de4609370cb7bb
# peer id 16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1
image: ghcr.io/reamlabs/ream:latest
privkey: "af27950128b49cda7e7bc9fcb7b0270f7a3945aa7543326f3bfdbd57d2a97a32"
enrFields:
#verify /ip4/127.0.0.1/udp/9001/quic-v1/p2p/16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1
Expand All @@ -30,6 +32,7 @@ validators:

- name: "qlean_0"
# TODO: add a third entry in nodes.yaml corresponding to this
image: qdrvm/qlean-mini:latest
privkey: "c2bbdac5e876b3e9d4b8b6b8c2bbdac5e876b3e9d4b8b6b8c2bbdac5e876b3e9"
enrFields:
#verify /ip4/127.0.0.1/udp/9001/quic-v1/p2p/16Uiu2HAmPQhkD6Zg5Co2ee8ShshkiY4tDePKFARPpCS2oKSLj1E1
Expand All @@ -41,6 +44,7 @@ validators:
- name: "lantern_0"
# node id a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2
# peer id 16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv
image: piertwo/lantern:v0.0.1
privkey: "d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5"
# verify /ip4/127.0.0.1/udp/9004/quic-v1/p2p/16Uiu2HAm7TYVs6qvDKnrovd9m4vvRikc4HPXm1WyLumKSe5fHxBv
enrFields:
Expand Down
28 changes: 23 additions & 5 deletions ansible/roles/grandine/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,30 @@
# Grandine role: Deploy and manage Grandine nodes
# Converts client-cmds/grandine-cmd.sh logic to Ansible tasks

- name: Extract docker image from client-cmd.sh
- name: Extract docker image from config file
shell: |
# Extract the first word (docker image) from node_docker line
# playbook_dir points to ansible/playbooks, go up two levels to reach project root
project_root="$(cd '{{ playbook_dir }}/../..' && pwd)"
grep -E '^node_docker=' "$project_root/client-cmds/grandine-cmd.sh" | head -1 | sed -E 's/.*node_docker="([^ "]+).*/\1/'
# Read image from user config file if provided, otherwise from validator-config.yaml
validator_config="{{ genesis_dir }}/validator-config.yaml"
user_config="{{ user_config_file | default('') }}"

# Try user config first if provided (supports both 'nodes' and 'clients' format)
if [ -n "$user_config" ] && [ -f "$user_config" ]; then
# Try nodes format first (grandine_0 style)
image=$(yq eval '.nodes[] | select(.name | test("^grandine_")) | .image' "$user_config" 2>/dev/null | head -1)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
# Try clients format (backwards compatibility)
image=$(yq eval '.clients[] | select(.name == "grandine") | .image' "$user_config" 2>/dev/null)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
fi

# Fall back to validator-config.yaml (read from validators array)
yq eval '.validators[] | select(.name | test("^grandine_")) | .image' "$validator_config" | head -1
register: grandine_docker_image_raw
changed_when: false
delegate_to: localhost
Expand Down
28 changes: 23 additions & 5 deletions ansible/roles/lantern/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,30 @@
# Lantern role: Deploy and manage Lantern nodes
# Converts client-cmds/lantern-cmd.sh logic to Ansible tasks

- name: Extract docker image from client-cmd.sh
- name: Extract docker image from config file
shell: |
# Extract the docker image from LANTERN_IMAGE line
# playbook_dir points to ansible/playbooks, go up two levels to reach project root
project_root="$(cd '{{ playbook_dir }}/../..' && pwd)"
grep -E '^LANTERN_IMAGE=' "$project_root/client-cmds/lantern-cmd.sh" | head -1 | sed -E 's/.*LANTERN_IMAGE="([^"]+)".*/\1/'
# Read image from user config file if provided, otherwise from validator-config.yaml
validator_config="{{ genesis_dir }}/validator-config.yaml"
user_config="{{ user_config_file | default('') }}"

# Try user config first if provided (supports both 'nodes' and 'clients' format)
if [ -n "$user_config" ] && [ -f "$user_config" ]; then
# Try nodes format first (lantern_0 style)
image=$(yq eval '.nodes[] | select(.name | test("^lantern_")) | .image' "$user_config" 2>/dev/null | head -1)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
# Try clients format (backwards compatibility)
image=$(yq eval '.clients[] | select(.name == "lantern") | .image' "$user_config" 2>/dev/null)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
fi

# Fall back to validator-config.yaml (read from validators array)
yq eval '.validators[] | select(.name | test("^lantern_")) | .image' "$validator_config" | head -1
register: lantern_docker_image_raw
changed_when: false
delegate_to: localhost
Expand Down
28 changes: 23 additions & 5 deletions ansible/roles/lighthouse/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,30 @@
# Lighthouse role: Deploy and manage Lighthouse nodes
# Converts client-cmds/lighthouse-cmd.sh logic to Ansible tasks

- name: Extract docker image from client-cmd.sh
- name: Extract docker image from config file
shell: |
# Extract the first word (docker image) from node_docker line
# playbook_dir points to ansible/playbooks, go up two levels to reach project root
project_root="$(cd '{{ playbook_dir }}/../..' && pwd)"
grep -E '^node_docker=' "$project_root/client-cmds/lighthouse-cmd.sh" | head -1 | sed -E 's/.*node_docker="([^ "]+).*/\1/'
# Read image from user config file if provided, otherwise from validator-config.yaml
validator_config="{{ genesis_dir }}/validator-config.yaml"
user_config="{{ user_config_file | default('') }}"

# Try user config first if provided (supports both 'nodes' and 'clients' format)
if [ -n "$user_config" ] && [ -f "$user_config" ]; then
# Try nodes format first (lighthouse_0 style)
image=$(yq eval '.nodes[] | select(.name | test("^lighthouse_")) | .image' "$user_config" 2>/dev/null | head -1)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
# Try clients format (backwards compatibility)
image=$(yq eval '.clients[] | select(.name == "lighthouse") | .image' "$user_config" 2>/dev/null)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
fi

# Fall back to validator-config.yaml (read from validators array)
yq eval '.validators[] | select(.name | test("^lighthouse_")) | .image' "$validator_config" | head -1
register: lighthouse_docker_image_raw
changed_when: false
delegate_to: localhost
Expand Down
28 changes: 23 additions & 5 deletions ansible/roles/qlean/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,30 @@
# Qlean role: Deploy and manage Qlean nodes
# Converts client-cmds/qlean-cmd.sh logic to Ansible tasks

- name: Extract docker image from client-cmd.sh
- name: Extract docker image from config file
shell: |
# Extract the first word (docker image) from node_docker line
# playbook_dir points to ansible/playbooks, go up two levels to reach project root
project_root="$(cd '{{ playbook_dir }}/../..' && pwd)"
grep -E '^node_docker=' "$project_root/client-cmds/qlean-cmd.sh" | head -1 | sed -E 's/.*node_docker="([^ "]+).*/\1/'
# Read image from user config file if provided, otherwise from validator-config.yaml
validator_config="{{ genesis_dir }}/validator-config.yaml"
user_config="{{ user_config_file | default('') }}"

# Try user config first if provided (supports both 'nodes' and 'clients' format)
if [ -n "$user_config" ] && [ -f "$user_config" ]; then
# Try nodes format first (qlean_0 style)
image=$(yq eval '.nodes[] | select(.name | test("^qlean_")) | .image' "$user_config" 2>/dev/null | head -1)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
# Try clients format (backwards compatibility)
image=$(yq eval '.clients[] | select(.name == "qlean") | .image' "$user_config" 2>/dev/null)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
fi

# Fall back to validator-config.yaml (read from validators array)
yq eval '.validators[] | select(.name | test("^qlean_")) | .image' "$validator_config" | head -1
register: qlean_docker_image_raw
changed_when: false
delegate_to: localhost
Expand Down
28 changes: 23 additions & 5 deletions ansible/roles/ream/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,30 @@
# Ream role: Deploy and manage Ream nodes
# Converts client-cmds/ream-cmd.sh logic to Ansible tasks

- name: Extract docker image from client-cmd.sh
- name: Extract docker image from config file
shell: |
# Extract the first word (docker image) from node_docker line
# playbook_dir points to ansible/playbooks, go up two levels to reach project root
project_root="$(cd '{{ playbook_dir }}/../..' && pwd)"
grep -E '^node_docker=' "$project_root/client-cmds/ream-cmd.sh" | head -1 | sed -E 's/.*node_docker="([^ "]+).*/\1/'
# Read image from user config file if provided, otherwise from validator-config.yaml
validator_config="{{ genesis_dir }}/validator-config.yaml"
user_config="{{ user_config_file | default('') }}"

# Try user config first if provided (supports both 'nodes' and 'clients' format)
if [ -n "$user_config" ] && [ -f "$user_config" ]; then
# Try nodes format first (ream_0 style)
image=$(yq eval '.nodes[] | select(.name | test("^ream_")) | .image' "$user_config" 2>/dev/null | head -1)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
# Try clients format (backwards compatibility)
image=$(yq eval '.clients[] | select(.name == "ream") | .image' "$user_config" 2>/dev/null)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
fi

# Fall back to validator-config.yaml (read from validators array)
yq eval '.validators[] | select(.name | test("^ream_")) | .image' "$validator_config" | head -1
register: ream_docker_image_raw
changed_when: false
delegate_to: localhost
Expand Down
31 changes: 22 additions & 9 deletions ansible/roles/zeam/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,30 @@
# Zeam role: Deploy and manage Zeam nodes
# Converts client-cmds/zeam-cmd.sh logic to Ansible tasks

- name: Extract docker image from client-cmd.sh
- name: Extract docker image from config file
shell: |
# Extract the docker image from node_docker line (find word containing / and :)
# playbook_dir points to ansible/playbooks, go up two levels to reach project root
project_root="$(cd '{{ playbook_dir }}/../..' && pwd)"
script_path="$project_root/client-cmds/zeam-cmd.sh"
if [ ! -f "$script_path" ]; then
echo "ERROR: Script not found at $script_path (project_root=$project_root, playbook_dir={{ playbook_dir }})" >&2
exit 1
# Read image from user config file if provided, otherwise from validator-config.yaml
validator_config="{{ genesis_dir }}/validator-config.yaml"
user_config="{{ user_config_file | default('') }}"

# Try user config first if provided (supports both 'nodes' and 'clients' format)
if [ -n "$user_config" ] && [ -f "$user_config" ]; then
# Try nodes format first (zeam_0 style)
image=$(yq eval '.nodes[] | select(.name | test("^zeam_")) | .image' "$user_config" 2>/dev/null | head -1)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
# Try clients format (backwards compatibility)
image=$(yq eval '.clients[] | select(.name == "zeam") | .image' "$user_config" 2>/dev/null)
if [ -n "$image" ] && [ "$image" != "null" ]; then
echo "$image"
exit 0
fi
fi
grep -E '^node_docker=' "$script_path" | head -1 | grep -oE '[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+:[a-zA-Z0-9._-]+' | head -1

# Fall back to validator-config.yaml (read from validators array)
yq eval '.validators[] | select(.name | test("^zeam_")) | .image' "$validator_config" | head -1
register: zeam_docker_image_raw
changed_when: false
delegate_to: localhost
Expand Down
5 changes: 4 additions & 1 deletion client-cmds/grandine-cmd.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
#!/bin/bash

# Docker image (set from validator-config.yaml or user config via --config-file)
# grandineImage is exported by spin-node.sh before sourcing this file

node_binary="$grandine_bin \
--genesis $configDir/config.yaml \
--validator-registry-path $configDir/validators.yaml \
Expand All @@ -10,7 +13,7 @@ node_binary="$grandine_bin \
--address 0.0.0.0 \
--hash-sig-key-dir $configDir/hash-sig-keys"

node_docker="sifrai/lean:devnet-2 \
node_docker="$grandineImage \
--genesis /config/config.yaml \
--validator-registry-path /config/validators.yaml \
--bootnodes /config/nodes.yaml \
Expand Down
5 changes: 3 additions & 2 deletions client-cmds/lantern-cmd.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
#!/bin/bash

#-----------------------lantern setup----------------------
LANTERN_IMAGE="piertwo/lantern:v0.0.2"
# Docker image (set from validator-config.yaml or user config via --config-file)
# lanternImage is exported by spin-node.sh before sourcing this file

devnet_flag=""
if [ -n "$devnet" ]; then
Expand All @@ -24,7 +25,7 @@ node_binary="$scriptDir/lantern/build/lantern_cli \
--log-level debug \
--hash-sig-key-dir $configDir/hash-sig-keys"

node_docker="$LANTERN_IMAGE --data-dir /data \
node_docker="$lanternImage --data-dir /data \
--genesis-config /config/config.yaml \
--validator-registry-path /config/validators.yaml \
--genesis-state /config/genesis.ssz \
Expand Down
5 changes: 4 additions & 1 deletion client-cmds/lighthouse-cmd.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
# Metrics enabled by default
metrics_flag="--metrics"

# Docker image (set from validator-config.yaml or user config via --config-file)
# lighthouseImage is exported by spin-node.sh before sourcing this file

node_binary="$lighthouse_bin lean_node \
--datadir \"$dataDir/$item\" \
--config \"$configDir/config.yaml\" \
Expand All @@ -16,7 +19,7 @@ node_binary="$lighthouse_bin lean_node \
--metrics-address 0.0.0.0 \
--metrics-port $metricsPort"

node_docker="hopinheimer/lighthouse:latest lighthouse lean_node \
node_docker="$lighthouseImage lighthouse lean_node \
--datadir /data \
--config /config/config.yaml \
--validators /config/validator-config.yaml \
Expand Down
Loading