Fix API key and model prompt integration with Devstral model#9
Fix API key and model prompt integration with Devstral model#9GuardianAngelWw wants to merge 3 commits intomainfrom
Conversation
|
|
WalkthroughA new integration guide was added, and both Docker Compose files were updated for the OpenHands frontend to support direct Devstral model integration with the Ollama backend. The changes automate API key handling, model selection, and connection by setting specific environment variables and switching to a prebuilt OpenHands image. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant OpenHands Frontend
participant Ollama Backend
participant Devstral Model
User->>OpenHands Frontend: Access UI
OpenHands Frontend->>OpenHands Frontend: Read env vars (API key, model, integration mode)
OpenHands Frontend->>Ollama Backend: Auto-connect using env vars
Ollama Backend->>Devstral Model: Route requests
Devstral Model-->>Ollama Backend: Respond
Ollama Backend-->>OpenHands Frontend: Respond
OpenHands Frontend-->>User: Display results (no API key/model prompt)
Possibly related PRs
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @GuardianAngelWw, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly improves the user experience for integrating the Devstral model with the OpenHands frontend. My changes eliminate unnecessary prompts for API keys and model selection by configuring direct model integration through new environment variables in the Docker Compose setups. This ensures a seamless and automated connection, making it easier for users to get started with the Devstral model.
Highlights
- Streamlined Devstral/OpenHands Integration: I've introduced several new environment variables (
LLM_API_KEY,SKIP_API_KEY_VALIDATION,AUTO_CONNECT_MODEL,DEFAULT_MODEL,DIRECT_MODEL_INTEGRATION) to Docker Compose configurations. These variables automate the API key and model selection process, allowing OpenHands to connect directly to the Devstral model without user prompts. - Docker Compose Configuration Updates: The new environment variables have been applied to both
ollama-setup/docker-compose.ymlandexamples/quick-start-ollama.ymlto ensure consistent and direct integration across different setup methods. Additionally, theopenhandsservice inollama-setup/docker-compose.ymlnow uses a pre-built image and has been renamed todevstral-openhandsfor clarity. - New Integration Documentation: A new documentation file,
docs/API_INTEGRATION.md, has been added. This document provides a comprehensive guide to the Devstral OpenHands integration, explaining the purpose of each new environment variable, implementation details, testing procedures, and troubleshooting tips.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request effectively addresses the integration issues by introducing necessary environment variables to bypass API key and model prompts for the Devstral/Ollama setup. The addition of the API_INTEGRATION.md document is valuable for explaining these changes and the required configuration. The changes in the Docker Compose files correctly apply these variables. I've provided a couple of medium-severity comments regarding potential clarity improvements around the documentation of the placeholder API key and the impact of changing the openhands service from build to image in the ollama-setup/docker-compose.yml file, which affects the user setup workflow.
| image: docker.all-hands.dev/all-hands-ai/openhands:0.40 | ||
| container_name: devstral-openhands |
There was a problem hiding this comment.
Changing from build to image means users of this specific docker-compose.yml will now pull a pre-built image instead of building the frontend locally. This simplifies setup for running the service but changes the workflow for users who might have been using this file for local frontend development. Consider adding a note about this change in the PR description or documentation if this is a significant shift in the intended use of this specific compose file.
|
|
||
| ### Explanation | ||
|
|
||
| 1. **LLM_API_KEY**: Provides a default API key, which prevents OpenHands from prompting the user for an API key. The value doesn't matter for Ollama integration, but must be present. |
There was a problem hiding this comment.
The documentation mentions the LLM_API_KEY value doesn't matter but must be present. While clear here, using a placeholder like sk-devstral-integration might be slightly confusing as it looks like a specific key format. Perhaps a more generic placeholder or explicitly stating 'any non-empty string will work' could enhance clarity, although the current explanation is also functional.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
examples/quick-start-ollama.yml (1)
35-40: Use a clear API key placeholder or.envfile for secrets
The hard-coded valuesk-devstral-integrationis meant as a dummy key but may be mistaken for a real secret. Consider replacing it with a placeholder likeYOUR_DEVSTRAL_API_KEYand loading it from a.envto avoid accidental leakage.docs/API_INTEGRATION.md (2)
3-3: Add missing article “the” for clarity
Change “integrates with OpenHands frontend” to “integrates with the OpenHands frontend” to improve readability.
14-18: Clarify placeholder usage for environment variables
These entries (especiallyLLM_API_KEY) are examples. Add a note that real values should be pulled from a.envfile or secrets manager, and update placeholders accordingly.ollama-setup/docker-compose.yml (1)
32-37: Centralize secret management and harmonize defaults
While these env vars enable direct integration, the dummy key may be misused. Consider sourcingLLM_API_KEYfrom a.envfile and, for consistency with the quick-start example, using fallback syntax (e.g.${DEFAULT_MODEL:-devstral}).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
docs/API_INTEGRATION.md(1 hunks)examples/quick-start-ollama.yml(1 hunks)ollama-setup/docker-compose.yml(3 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/API_INTEGRATION.md
[uncategorized] ~3-~3: You might be missing the article “the” here.
Context: ... how the Devstral model integrates with OpenHands frontend, particularly focusing on the ...
(AI_EN_LECTOR_MISSING_DETERMINER_THE)
🪛 markdownlint-cli2 (0.17.2)
docs/API_INTEGRATION.md
51-51: Bare URL used
null
(MD034, no-bare-urls)
🔇 Additional comments (2)
ollama-setup/docker-compose.yml (2)
22-23: Approve: Switched to prebuilt OpenHands image
Usingdocker.all-hands-ai/openhands:0.40and renaming the container todevstral-openhandsensures version consistency and removes the need for a local build.
56-56: Approve: Customized Web UI name
SettingWEBUI_NAME=Devstral Model Managerprovides a clearer label for users managing multiple interfaces.
1 participant
Description
This PR fixes the integration issues between the Devstral backend (Ollama) and the OpenHands frontend, particularly focusing on the API key and model prompt handling.
Problem
Without the correct environment variables, users were being prompted for API keys or model selection even though direct integration was intended. This created unnecessary friction in the user experience when the system was designed to work with the Devstral model directly.
Solution
Added several environment variables to the Docker Compose configurations to properly handle API keys and model prompts:
LLM_API_KEY: Provides a default API key to skip the promptSKIP_API_KEY_VALIDATION: Bypasses API key validationAUTO_CONNECT_MODEL: Automatically connects to the model without promptingDEFAULT_MODEL: Sets default model to useDIRECT_MODEL_INTEGRATION: Enables direct model integration pathChanges Made
ollama-setup/docker-compose.ymlwith the new environment variablesexamples/quick-start-ollama.ymlwith the same configurationdocs/API_INTEGRATION.mdexplaining the changes and how they solve the issueTesting
The changes have been tested to ensure OpenHands now connects directly to the Devstral model without prompting for API keys or model selection.
Related Issues
Resolves the integration issues between Devstral backend and OpenHands frontend.
Summary by CodeRabbit
Documentation
Chores