[RHIDP-11652] Add BYOM Architecture for Lightspeed

[Jdubrick]

Description

• Removes default ollama and instead is full BYOM for Lightspeed
• Updates Lightspeed Core container image
• Updates Lightspeed plugins
• Adds sync script for config files (similar to Operator and Helm)
  • Will get the same auto PRs for updating this content when Operator and Helm do to reduce manual runs
• Updates start/stop scripts to reflect new validation handling
• Updates readme for instructions on how to add LLM(s)

Note: There will be a follow up PR to have Lightspeed included by default, these changes are in preparation for that.

Which issue(s) does this PR fix or relate to

https://redhat.atlassian.net/browse/RHIDP-11652

PR acceptance criteria

• Tests updated and passing
• Documentation updated
• Built-in TechDocs updated if needed. Note that TechDocs changes may need to be reviewed by a Product Manager and/or Architect to ensure content accuracy, clarity, and alignment with user needs.

How to test changes / Special notes to the reviewer

Follow the instructions in the README.

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

1. Ollama URL missing /v1 ☑ 🐞 Bug ≡ Correctness

Description

The README configures OLLAMA_URL without /v1, but config.yaml uses OLLAMA_URL directly as
the OpenAI-compatible base URL (defaulting to .../v1), so users following the README example will
point to the wrong endpoint and inference calls may fail.

Code

developer-lightspeed/README.md[R149-154]

+    # REQUIRED: URL to your Ollama server
+    # Examples:
+    #   - Local Ollama: http://host.docker.internal:11434
+    #   - Remote Ollama: https://your-ollama-server.com:11434
+    OLLAMA_URL=http://host.docker.internal:11434
+    ```

Relevance

⭐⭐⭐ High

Config default expects OpenAI-compatible /v1; README omits it, likely misleads users. Similar
endpoint conventions in past.

PR-#111
PR-#126
PR-#163

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The README instructs OLLAMA_URL=http://host.docker.internal:11434 (no /v1). The Lightspeed Core
runtime config uses OLLAMA_URL as base_url and its default explicitly includes /v1, indicating
the expected value is the OpenAI-compatible /v1 endpoint.

developer-lightspeed/README.md[149-154]
developer-lightspeed/configs/extra-files/config.yaml[49-53]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`developer-lightspeed/README.md` documents an `OLLAMA_URL` value that omits the `/v1` path, but the runtime config consumes `OLLAMA_URL` as an OpenAI-compatible base URL (default includes `/v1`). This mismatch will cause users to configure an endpoint that likely fails.

### Issue Context
- `remote::ollama` provider is configured with `base_url: ${env.OLLAMA_URL:=http://localhost:11434/v1}`.
- The README’s example should match the expected OpenAI-compatible endpoint.

### Fix Focus Areas
- developer-lightspeed/README.md[149-154]
- developer-lightspeed/configs/extra-files/config.yaml[49-53]

### What to change
- Update the README example(s) to include `/v1` (e.g., `http://host.docker.internal:11434/v1`).
- Optionally add a short note that the URL must point at the OpenAI-compatible `/v1` endpoint.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Vertex credentials path unusable ☑ 🐞 Bug ☼ Reliability

Description

The README/default.env instruct setting GOOGLE_APPLICATION_CREDENTIALS to an absolute JSON path,
but the provided developer-lightspeed/compose.yaml does not mount any credentials file into the
lightspeed-core container, so the container cannot read that host path and Vertex AI auth will
fail.

Code

developer-lightspeed/README.md[R168-174]

    ENABLE_VERTEX_AI=true
    
    # REQUIRED: Absolute path to your Google Cloud credentials JSON file
-    VERTEX_AI_CREDENTIALS_PATH=/absolute/path/to/your/google-cloud-credentials.json
+    GOOGLE_APPLICATION_CREDENTIALS=/absolute/path/to/your/google-cloud-credentials.json
    
    # REQUIRED: Your GCP project ID
    VERTEX_AI_PROJECT=your-gcp-project-id

Relevance

⭐⭐⭐ High

Previously repo mounted Vertex credentials into container; now README uses host path without mount,
causing auth failure.

PR-#130
PR-#152
PR-#163

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

Both README and default.env describe GOOGLE_APPLICATION_CREDENTIALS as an absolute path to a
local service-account JSON file. However, the Lightspeed Core container only mounts the
stack/config/profile files and RAG volumes; there is no bind mount for the credentials JSON, so the
referenced path will not exist inside the container by default.

developer-lightspeed/README.md[166-174]
default.env[97-106]
developer-lightspeed/compose.yaml[34-53]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Vertex AI setup instructs users to set `GOOGLE_APPLICATION_CREDENTIALS` to an absolute path of a host JSON file, but the provided compose stack does not mount that file into the container. This makes the documented setup non-functional.

### Issue Context
In containerized execution, `GOOGLE_APPLICATION_CREDENTIALS` must point to a file path that exists *inside the container*. Currently, `lightspeed-core` mounts only:
- `/app-root/lightspeed-stack.yaml`
- `/app-root/rhdh-profile.py`
- `/app-root/config.yaml`
- RAG volumes

### Fix Focus Areas
- developer-lightspeed/compose.yaml[34-53]
- developer-lightspeed/README.md[166-187]
- default.env[97-106]

### Fix options (pick one)
1) **Compose-based solution (recommended)**
  - Add a credentials bind-mount to `lightspeed-core` (e.g., mount a host file path from an env var into `/app-root/credentials.json`).
  - Set `GOOGLE_APPLICATION_CREDENTIALS=/app-root/credentials.json` inside the container (via `environment:`), and update docs to match.

2) **Docs-only solution**
  - Update README/default.env to explicitly instruct users to add a `compose.override.yaml` (or similar) that bind-mounts the JSON into the container, and set `GOOGLE_APPLICATION_CREDENTIALS` to the in-container path.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[rhdh-qodo-merge]

Review Summary by Qodo

Implement BYOM Architecture for Developer Lightspeed

✨ Enhancement 📝 Documentation

Walkthroughs

Description

• Removes Ollama default and implements full BYOM architecture
• Simplifies start/stop scripts by removing interactive provider selection
• Adds sync script for upstream Lightspeed config file management
• Updates Lightspeed Core to library mode with unified container
• Replaces safety guard configuration with query validation feature
• Updates environment variables and documentation for BYOM setup

Diagram

flowchart LR
  A["Old Architecture<br/>Ollama Default +<br/>Llama Stack"] -->|Remove| B["BYOM Architecture<br/>No Default Provider"]
  B -->|Configure| C["External Providers<br/>vLLM, OpenAI,<br/>Ollama, Vertex AI"]
  D["Interactive Scripts<br/>Provider Selection"] -->|Simplify| E["Simplified Scripts<br/>Runtime Detection Only"]
  F["Multiple Compose Files<br/>with-ollama,<br/>with-safety-guard"] -->|Consolidate| G["Single Compose File<br/>Library Mode"]
  H["Llama Stack Separate"] -->|Merge| I["Lightspeed Core<br/>Library Mode"]
  J["Safety Guard Config"] -->|Replace| K["Query Validation<br/>Optional Feature"]

Loading

File Changes

1. developer-lightspeed/scripts/start-lightspeed.sh ✨ Enhancement +25/-141

Simplify setup script to remove provider selection

developer-lightspeed/scripts/start-lightspeed.sh

---

2. developer-lightspeed/scripts/stop-lightspeed.sh ✨ Enhancement +48/-243

Simplify cleanup script removing config detection logic

developer-lightspeed/scripts/stop-lightspeed.sh

---

3. developer-lightspeed/scripts/sync-lightspeed-configs.sh ✨ Enhancement +138/-0

Add new sync script for upstream config management

developer-lightspeed/scripts/sync-lightspeed-configs.sh

---

View more (16)

4. .github/workflows/test.yml ⚙️ Configuration changes +1/-5

Update CI to test single Lightspeed configuration

.github/workflows/test.yml

---

5. compose.yaml ⚙️ Configuration changes +2/-0

Add MAX_ENTRY_SIZE to plugin installer environment

compose.yaml

---

6. default.env ⚙️ Configuration changes +39/-36

Restructure env vars for BYOM provider configuration

default.env

---

7. developer-lightspeed/README.md 📝 Documentation +165/-257

Rewrite documentation for BYOM architecture

developer-lightspeed/README.md

---

8. developer-lightspeed/compose-with-ollama.yaml ⚙️ Configuration changes +0/-118

Remove Ollama-specific compose configuration file

developer-lightspeed/compose-with-ollama.yaml

---

9. developer-lightspeed/compose-with-safety-guard-ollama.yaml ⚙️ Configuration changes +0/-30

Remove Ollama safety guard compose configuration

developer-lightspeed/compose-with-safety-guard-ollama.yaml

---

10. developer-lightspeed/compose-with-safety-guard.yaml ⚙️ Configuration changes +0/-49

Remove BYO safety guard compose configuration

developer-lightspeed/compose-with-safety-guard.yaml

---

11. developer-lightspeed/compose.yaml ✨ Enhancement +21/-50

Consolidate to single compose with Lightspeed Core library mode

developer-lightspeed/compose.yaml

---

12. developer-lightspeed/configs/dynamic-plugins/dynamic-plugins.lightspeed.yaml Dependencies +4/-2

Update Lightspeed plugin versions to 2.2.1

developer-lightspeed/configs/dynamic-plugins/dynamic-plugins.lightspeed.yaml

---

13. developer-lightspeed/configs/extra-files/config.yaml ⚙️ Configuration changes +226/-0

Add new Lightspeed Core configuration with validation support

developer-lightspeed/configs/extra-files/config.yaml

---

14. developer-lightspeed/configs/extra-files/lightspeed-stack.yaml ⚙️ Configuration changes +30/-9

Update to library mode configuration with profile support

developer-lightspeed/configs/extra-files/lightspeed-stack.yaml

---

15. developer-lightspeed/configs/extra-files/rhdh-profile.py ⚙️ Configuration changes +268/-0

Add new profile with system prompts and validation logic

developer-lightspeed/configs/extra-files/rhdh-profile.py

---

16. developer-lightspeed/configs/extra-files/run-no-guard.yaml ⚙️ Configuration changes +0/-160

Remove legacy Llama Stack configuration file

developer-lightspeed/configs/extra-files/run-no-guard.yaml

---

17. developer-lightspeed/configs/extra-files/run.yaml ⚙️ Configuration changes +0/-187

Remove legacy Llama Stack safety guard configuration

developer-lightspeed/configs/extra-files/run.yaml

---

18. developer-lightspeed/configs/extra-files/templates/placeholder.json Miscellaneous +0/-1

Remove unused placeholder credentials file

developer-lightspeed/configs/extra-files/templates/placeholder.json

---

19. orchestrator/compose.yaml ⚙️ Configuration changes +0/-2

Move MAX_ENTRY_SIZE to main compose plugin installer

orchestrator/compose.yaml

---

[Jdubrick]

/cc @rm3l

[gabemontero]

didn't pick up on this @Jdubrick if it occurred with the operator/helm chart work, but with the removal of the run-no-guard.yaml and run.yaml as part of running LLS as a service, do we no longer provide a means of choosing between guard and no-guard ?

thanks

[Jdubrick]

didn't pick up on this @Jdubrick if it occurred with the operator/helm chart work, but with the removal of the run-no-guard.yaml and run.yaml as part of running LLS as a service, do we no longer provide a means of choosing between guard and no-guard ?

thanks

@gabemontero yeah we now enable/disable validation based on an env var. It's documented across the implementations and also will be in the official rhdh docs for lightspeed. You can see the blurb related to rhdh local here: https://github.com/Jdubrick/rhdh-local/blob/remove-lightspeed-ollama/developer-lightspeed/README.md?plain=1#L190

[gabemontero]

didn't pick up on this @Jdubrick if it occurred with the operator/helm chart work, but with the removal of the run-no-guard.yaml and run.yaml as part of running LLS as a service, do we no longer provide a means of choosing between guard and no-guard ?
thanks

@gabemontero yeah we now enable/disable validation based on an env var. It's documented across the implementations and also will be in the official rhdh docs for lightspeed. You can see the blurb related to rhdh local here: https://github.com/Jdubrick/rhdh-local/blob/remove-lightspeed-ollama/developer-lightspeed/README.md?plain=1#L190

cool thanks for the explanation / pointer @Jdubrick

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[gabemontero]

forgot to mention ... I had no other comments of note, lgtm (though yep @rm3l had a good catch on the default item I see you already addressed)

But yeah I will defer to @rm3l and the like to apply lgtms / approvals

thanks