Skip to content

Linux_x86_64_cpu_only_Ubuntu_ServerConfig_update

Choose a tag to compare

View all tags
@7ZoneSystems 7ZoneSystems released this 06 Mar 00:17
· 120 commits to main since this release
update
45f02c6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Native Lab Pro — Release Notes

v2.1.0 · Server & Binary Configuration

March 2026 · Additive, non-breaking change · 1 new tab · 9 change sets

Overview

v2.1.0 introduces a dedicated 🖥️ Server & Binary Configuration tab, giving users full control over llama-cli and llama-server binary paths without hardcoding. Previously, binary locations were resolved at startup from a fixed directory structure. This update makes those paths user-configurable, adds per-OS detection, persists settings to disk, and plumbs the configuration through every inference code path.

Motivation

The previous approach had several limitations:

  • Binaries had to live in a fixed relative path (./llama/bin/) — no flexibility for system-wide installs or custom builds
  • Windows, macOS, and Linux users needed to manually edit source to point to the correct executable
  • The llama-server host and port range were hardcoded to 127.0.0.1:8600–8700 with no way to change them
  • Extra launch flags (e.g. --numa, --flash-attn, --no-mmap) required source edits
  • There was no in-app way to verify a binary was present and functional before loading a model

What's New

🖥️ Server & Binary Configuration Tab

A new tab added to the main interface alongside Models, Config, Logs, and Appearance. It contains four sections:

  • Binary Paths — browse for llama-cli and llama-server executables with live ✅/❌ file-exists indicators
  • Server Settings — configure bind host and port scan range used when starting llama-server instances
  • Extra Launch Flags — append arbitrary flags to every CLI or server launch without touching source code
  • Binary Test — run --version against either binary and see the output inline to verify it works

ServerConfig Dataclass & Persistence

A new ServerConfig dataclass stores all settings and serialises them to localllm/server_config.json. Settings survive restarts and are loaded at app startup. The class also exposes detected_os, default_cli_name, and default_server_name properties that adapt to Windows, macOS, and Linux automatically.

Dynamic Binary Resolution

A _resolve_binary() helper and _refresh_binary_paths() function update the module-level LLAMA_CLI and LLAMA_SERVER variables whenever settings are saved. All existing inference code paths — CLI workers, server launch, pipeline mode, chunked summary, and multi-PDF — automatically pick up the new paths without any further changes.

Configurable Port Range

_free_port() now reads port_range_lo and port_range_hi from ServerConfig instead of using hardcoded 8600/8700. Defaults remain unchanged, so existing setups are unaffected.

Detailed Change Log

| Location | Description | Type

-- | -- | -- | --
1 | SERVER_CONFIG_FILE | New constant pointing to localllm/server_config.json | ADDED
2 | ServerConfig dataclass | Stores cli_path, server_path, host, port ranges, extra flags. Includes save(), load(), and OS detection. | ADDED
3 | _resolve_binary() | Helper that returns a custom path if set and valid, otherwise the built-in default | ADDED
4 | _refresh_binary_paths() | Updates module-level LLAMA_CLI / LLAMA_SERVER from ServerConfig at runtime | ADDED
5 | LLAMA_CLI / LLAMA_SERVER | Renamed default constants to _LLAMA_CLI_DEFAULT / _LLAMA_SERVER_DEFAULT. Live vars now resolve through _refresh_binary_paths() | CHANGED
6 | _free_port() | Default lo/hi params changed to 0; reads ServerConfig.port_range_lo/hi at call time | CHANGED
7 | LlamaEngine._start_server() | Reads SERVER_CONFIG.host and extra_server_args; appends extra flags to the launch command | CHANGED
8 | LlamaEngine.create_worker() | CLI branch appends SERVER_CONFIG.extra_cli_args to the llama-cli command list | CHANGED
9 | ServerTab (UI class) | New widget — binary browse, server settings, flag editor, test runner. Saved via ServerConfig. | ADDED
10 | MainWindow._build_ui() | Registers ServerTab as new tab between Config and Logs | CHANGED
11 | MainWindow._toggle_theme() | Rebuilds ServerTab on theme switch to pick up updated palette colours | CHANGED

Files Changed

native_lab_pro.py      (+~420 lines added, ~30 lines modified)
localllm/server_config.json   (new — created on first save)

Migration & Compatibility

This release is fully backwards-compatible. No action required for existing installations:

  • If server_config.json does not exist, all defaults match previous behaviour exactly
  • LLAMA_CLI and LLAMA_SERVER resolve to the same built-in paths as before unless overridden
  • Port range defaults remain 8600–8700, host defaults remain 127.0.0.1
  • No database migrations, no session format changes, no breaking API changes

How to Use

  1. Open the app and navigate to the new 🖥️ Server tab
  2. Under Binary Paths, click Browse… next to llama-cli and select your binary
  3. Repeat for llama-server
  4. Optionally adjust the bind host, port range, and any extra launch flags
  5. Click Test to verify each binary responds correctly
  6. Click Save Server Settings — written to localllm/server_config.json
  7. Reload your model via Models tab or Model > Reload Model

The ✅ / ❌ indicators next to each path field update in real-time as you type or browse, showing whether the file exists before you save.

Known Limitations

  • Extra flags are passed as a whitespace-split string — flags with spaces in their values are not supported
  • The Test button uses --version, which not all llama.cpp builds support; some may exit non-zero but still be functional
  • Changing the port range does not affect already-running server instances — only new launches pick up the updated range

Upcoming

  • Per-role binary overrides (e.g. a separate CUDA build for the coding engine)
  • GPU layer configuration (--n-gpu-layers) exposed in the Server tab
  • Auto-discovery scan that searches common install paths and populates fields automatically
  • Quoted-argument support for extra flag fields
Assets 6
Loading