diff --git a/README.md b/README.md index be3d71e..251b030 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ This project reproduces the legacy PHP/jQuery stack with a contemporary architec ## How This Differs From The Original -- The original UmpleOnline in the legacy `umple` repo is a PHP/jQuery application; this repo splits the app into a React/Vite frontend, a Go API, and a Node-based code execution service. +- The original UmpleOnline in the legacy `umple` repo is a PHP/jQuery application; this repo splits the app into a React/Vite frontend, a Go API, dedicated collaboration and LSP proxy services, and a Node-based code execution service. - Local development is container-first: `make dev` brings up the backend services and the frontend hot-reload server, instead of the older mixed script-based setup. - The `code-exec` service defaults to port `4401`, while the legacy `UmpleCodeExecution` service defaults to `4400`, so both stacks can run on the same machine without a port collision. - Backend, collab, LSP, and code-exec ports can all be remapped from the repo root `.env` when you need to run multiple local stacks side by side. @@ -26,7 +26,7 @@ This project reproduces the legacy PHP/jQuery stack with a contemporary architec │ CodeMirror · ReactFlow │ │ Tailwind CSS · Zustand │ └────────────┬────────────────┘ - │ /api/* + │ /api/* · /ws/collab/* · /ws/lsp ┌────────────▼────────────────┐ │ Backend (Go/Chi) │ Port 3001 │ ├─ TCP ──▶ umplesync.jar │ @@ -37,13 +37,23 @@ This project reproduces the legacy PHP/jQuery stack with a contemporary architec │ Code Exec (Node.js) │ Port 4401 │ Sandboxed code runner │ └─────────────────────────────┘ + +┌─────────────────────────────┐ +│ Collab (Yjs WebSocket) │ Port 3002 +└─────────────────────────────┘ + +┌─────────────────────────────┐ +│ LSP Proxy (WS to stdio) │ Port 9999 +└─────────────────────────────┘ ``` -Three services, all containerized with Docker: +Five services, all containerized with Docker: -- **Frontend** — React 19, TypeScript, Vite, CodeMirror 6, ReactFlow -- **Backend** — Go 1.24 (Chi router), communicates with `umplesync.jar` via TCP and Graphviz for diagram rendering -- **Code Exec** — Node.js service for running compiled code in a sandbox +- **Frontend** - React 19, TypeScript, Vite, CodeMirror 6, ReactFlow. See [frontend/README.md](frontend/README.md). +- **Backend** - Go 1.24 (Chi router), communicates with `umplesync.jar` via TCP and Graphviz for diagram rendering. See [backend/README.md](backend/README.md). +- **Collab** - Yjs WebSocket server for realtime shared editing. See [collab/README.md](collab/README.md). +- **LSP Proxy** - WebSocket-to-stdio bridge for `umple-lsp-server`. See [lsp-proxy/README.md](lsp-proxy/README.md). +- **Code Exec** - Node.js service for running compiled code in a sandbox. See [code-exec/README.md](code-exec/README.md). ## Getting Started diff --git a/backend/README.md b/backend/README.md new file mode 100644 index 0000000..32eae9b --- /dev/null +++ b/backend/README.md @@ -0,0 +1,133 @@ +# Backend API + +The backend is the Go API layer for UmpleOnline. It owns model persistence, +compiler orchestration, diagram/export generation, examples, tasks, status +reporting, AI provider proxying, and the bridge to the code execution service. + +## Runtime Shape + +``` +Browser / frontend nginx + | + | /api/* + v +Go backend (Chi) + | + | TCP + v +umplesync.jar server + +Go backend + | + | HTTP + v +code-exec service + +Go backend + | + | filesystem + v +/data/models, /examples, /jars +``` + +The backend listens on `PORT`, defaulting to `3001`. In development Vite +proxies `/api/*` to it. In production the frontend nginx container proxies +`/api/*` to the backend container. + +## Internal Architecture + +- `cmd/server/main.go` loads configuration, initializes stores, starts the + compiler pool, wires the router, and handles graceful shutdown. +- `internal/config` maps environment variables to runtime config. +- `internal/api/router.go` registers all HTTP routes under `/api`. +- `internal/api/handlers` contains request handlers for generation, diagrams, + exports, execution, examples, models, CRUD helpers, tasks, health, and status. +- `internal/compiler` starts and supervises `umplesync.jar` as a long-running + TCP server, sends compiler commands, and serializes writes per model. +- `internal/model` stores model directories, tab metadata, `.ump` tab files, + generated assets, and temporary-model cleanup. +- `internal/task` stores task definitions and responses on disk. +- `internal/execution` proxies run requests to `code-exec`. + +The backend uses filesystem storage by design. Model IDs map to directories +under `MODEL_STORE_PATH`, and multi-tab models are represented as `tabs.json` +plus one `.ump` file per tab. + +## Main API Families + +- `GET /api/health` and `GET /api/status` - service and dependency health. +- `POST /api/generate` - compile and generate a target output. +- `POST /api/sync` - apply text/diagram synchronization operations. +- `POST /api/diagram` and `POST /api/export` - diagram rendering and export. +- `POST /api/execute` - compile and run generated Java/Python through + `code-exec`. +- `GET /api/models/{id}` and `POST /api/models/{id}/promote` - model loading + and temporary-to-durable promotion. +- `GET /api/examples`, `GET /api/examples/{id}`, and + `GET /api/examples/resolve` - bundled example catalog. +- `POST /api/crud/schema` and `POST /api/crud/diagram` - CRUD model helpers. +- `POST /api/tasks` and related task response routes - task workflows. +- `/api/ai/*` - provider proxying for browser-origin AI calls. + +## Compiler Lifecycle + +`internal/compiler.Pool` starts: + +```text +java -cp $UMPLE_SYNC_JAR cruise.umple.PlaygroundMain -server $UMPLE_PORT +``` + +Requests connect to that server over TCP and send the command format expected +by `umplesync.jar`. If the JVM exits or a TCP dial fails, the pool attempts a +restart. Per-model mutexes prevent concurrent writes to the same model +directory while compile/sync flows are active. + +## How This Differs From The Original + +Legacy UmpleOnline handled most backend work in PHP. The closest original +entry points are: + +- `~/umple/umpleonline/umple.php` - rendered the page, interpreted URL + parameters, loaded examples/models, and emitted runtime configuration. +- `~/umple/umpleonline/scripts/compiler.php` - handled AJAX actions for saving, + loading, compiling, image/export generation, and task operations. +- `~/umple/umpleonline/scripts/compiler_config.php` - contained the DataStore + abstraction and low-level socket calls to the compiler. + +This backend keeps the compiler contract but changes the architecture: + +- HTTP behavior is split into typed Go handlers instead of one large + request-switching PHP script. +- Model storage is explicit Go code under `internal/model`, not PHP helper + functions embedded in compiler configuration. +- Compiler process supervision lives in `internal/compiler.Pool`, with restart + and per-model locking in one place. +- Examples are served from the checked-in `examples/` snapshot instead of being + assembled from legacy page/menu generation at runtime. +- Code execution is delegated to the separate `code-exec` service instead of + being handled inside the same PHP endpoint family. +- Status and health are first-class JSON APIs rather than operational details + hidden behind legacy script pages. + +## Development + +From the repo root: + +```bash +make dev-backend +make logs-backend +make tidy +``` + +Focused Go checks should run from this folder: + +```bash +go test ./... +go vet ./... +``` + +When adding Go dependencies in the Docker dev stack, run: + +```bash +docker compose exec backend go mod tidy +``` diff --git a/collab/README.md b/collab/README.md new file mode 100644 index 0000000..febefff --- /dev/null +++ b/collab/README.md @@ -0,0 +1,108 @@ +# Collaboration Service + +This service owns realtime multi-user editing for UmpleOnline. It is a small +Node/TypeScript WebSocket server that speaks the Yjs sync and awareness +protocols used by the React frontend. + +## Runtime Shape + +``` +Browser editor + | + | y-websocket over /ws/collab/{roomId} + v +collab service + | + | in-memory Y.Doc per room + v +connected browser peers +``` + +The frontend connects through the same origin as the app: + +- Dev: Vite proxies `/ws/collab/*` to `localhost:${COLLAB_PORT:-3002}`. +- Prod: nginx in the frontend container proxies `/ws/collab/*` to the + `collab` container. + +The service exposes: + +- `GET /health` - plain `ok` health check for Docker Compose. +- `GET /status` - JSON runtime stats for active rooms, connections, awareness + states, process ID, uptime, and connection counters. +- `WS /ws/collab/{roomId}` - Yjs sync and awareness transport. + +## Internal Architecture + +- `src/server.ts` creates the HTTP server, WebSocket server, health endpoint, + status endpoint, and room routing. +- `src/sync.ts` contains the Yjs document registry and WebSocket protocol + handling. +- Each room ID maps to one in-memory `WSSharedDoc`. +- Each `WSSharedDoc` tracks connected sockets plus Yjs awareness client IDs. +- Document updates are rebroadcast with `y-protocols/sync`. +- Presence updates are rebroadcast with `y-protocols/awareness`. +- When the last socket leaves a room, the Yjs document is destroyed and removed + from memory. + +The service does not persist collaboration state. The room is a live shared +editing session. Durable model state is still managed by the backend model +store when the frontend compiles, syncs, promotes, or loads a model. + +## Frontend Contract + +The React frontend owns the user-facing collaboration lifecycle: + +- `frontend/src/hooks/useCollab.ts` creates the `Y.Doc`, connects the + `WebsocketProvider`, sets awareness identity, and hydrates or seeds shared + tabs. +- `frontend/src/hooks/useCollabTabs.ts` maps shared Yjs tab state into the + local Zustand session store. +- `frontend/src/components/editor/UmpleEditor.tsx` binds CodeMirror to the + active shared `Y.Text`. + +Shared Yjs data uses these names: + +- `Y.Map("tabs")` stores tab metadata by tab ID. +- `Y.Text("tab:")` stores each tab's source text. + +## How This Differs From The Original + +Legacy UmpleOnline handled collaboration inside the PHP/jQuery page flow. The +old UI in `~/umple/umpleonline/umple.php` rendered collaboration controls, +loaded `scripts/socket.io/socket.io.js`, and called browser globals from +`scripts/umpleCollab.js`. Its server endpoint was configured through +`scripts/collab-server-config.js.template` with a Socket.IO path such as +`/collabapi`. + +This rewrite separates collaboration into a dedicated service: + +- The collaboration transport is Yjs binary sync over WebSocket, not the legacy + Socket.IO collaboration client. +- Room state is isolated in `collab/src/sync.ts` instead of being mixed into + `Page`, `Action`, and global browser objects. +- The frontend stores collaborative tabs in structured Yjs maps/text objects + instead of patching a single page-level text model. +- The service has its own health and status endpoints, so orchestration and + monitoring do not depend on PHP page rendering. + +## Development + +From this folder: + +```bash +bun install +bun run dev +bun run build +``` + +Normally this service is started by Docker Compose: + +```bash +make dev-backend +``` + +or as part of the full stack: + +```bash +make dev +``` diff --git a/frontend/README.md b/frontend/README.md new file mode 100644 index 0000000..7b9e39f --- /dev/null +++ b/frontend/README.md @@ -0,0 +1,118 @@ +# Frontend + +The frontend is the browser application for UmpleOnline. It is a React 19, +TypeScript, Vite, Tailwind CSS v4, CodeMirror 6, ReactFlow, and Zustand app. + +## Runtime Shape + +``` +Browser + | + | React SPA + v +CodeMirror editor + diagram/output panels + | + | /api/* -> Go backend + | /ws/collab/{roomId} -> collab service + | /ws/lsp?session=... -> lsp-proxy service + v +backend services +``` + +In development, Vite serves the app on port `3200` and proxies API/WebSocket +traffic. In production, nginx serves the built SPA on port `80` inside the +frontend container and proxies backend, collaboration, and LSP traffic to the +other containers. + +## Internal Architecture + +- `src/main.tsx` configures routes and mounts the app. +- `src/App.tsx` provides global UI shell concerns such as theme setup, + document title, tooltips, and toasts. +- `src/components/layout/AppShell.tsx` composes the main editor workspace: + sidebar, toolbar, editor panel, output panel, diagram panel, command palette, + onboarding, and task sheet. +- `src/components/editor` contains CodeMirror editor surfaces, tabs, + generation/output panels, execution output, and selection tools. +- `src/components/diagram` renders class/state/other diagram output, including + SVG interaction and diagram toolbar controls. +- `src/api` centralizes calls to the Go backend. +- `src/hooks` contains lifecycle orchestration for compiler calls, URL loading, + collaboration, LSP, examples, execution, diagram sync, and task routes. +- `src/stores` contains Zustand stores for persisted session data, preferences, + ephemeral UI state, collaboration state, CRUD state, and task state. +- `src/codemirror` contains the Umple CodeMirror language support and LSP + client integration. +- `src/generation` centralizes generation target definitions. +- `src/pages/status` provides the developer status page. + +## State Model + +The frontend intentionally separates user-visible model state from transient UI +state: + +- `sessionStore` is the main editor/model store: current code, tabs, active + model ID, generation target, diagram caches, parsed model data, filters, and + chat messages. +- `preferencesStore` holds persistent UI preferences such as theme, panel + choices, diagram options, and sidebar state. +- `ephemeralStore` holds short-lived UI state such as loading/error flags, + output view state, and active panel choices. +- `collabStore` holds collaboration connection state and room metadata, while + the actual Yjs objects live outside Zustand. + +Compile and diagram generation are orchestrated by `useCompiler`, which reads +the current session snapshot and calls the backend through `src/api/client.ts`. + +## Service Integration + +- Backend API calls use relative `/api` URLs so dev and prod can share the same + browser-facing contract. +- Collaboration uses `y-websocket` against `/ws/collab/{roomId}` and stores + shared tabs in Yjs. +- LSP uses `@codemirror/lsp-client` against `/ws/lsp`, with one + `umple-lsp-server` process spawned by `lsp-proxy` for the active model. +- Diagram SVG edits can write back to the local session store or the shared + Yjs tab, depending on whether collaboration is active. + +## How This Differs From The Original + +Legacy UmpleOnline rendered the main page from +`~/umple/umpleonline/umple.php`, then coordinated the UI through global +JavaScript objects such as `Page`, `Action`, `Layout`, and `TabControl` in +`scripts/umple_page.js`, `scripts/umple_action.js`, and related files. AJAX +calls were made directly to `scripts/compiler.php`. + +This frontend changes that shape: + +- The UI is a typed React component tree instead of server-rendered PHP plus + global jQuery handlers. +- State is centralized in Zustand stores instead of page-wide mutable globals. +- API access is centralized in `src/api/client.ts` instead of scattered + `Ajax.sendRequest("scripts/compiler.php", ...)` calls. +- CodeMirror 6 is integrated as a React editor surface with explicit hooks for + compile, collaboration, and LSP lifecycles. +- URL/model loading, examples, generation targets, task routes, and status UI + are separate modules rather than branches inside a monolithic page script. +- Styling uses Tailwind v4 semantic tokens from `src/index.css` instead of + inline PHP page styles and jQuery UI-era button styling. + +## Development + +From this folder: + +```bash +bun install +bun run dev +bun run build +bun run test +bun run test:e2e +``` + +From the repo root, the usual entry points are: + +```bash +make dev-frontend +make test-e2e +make check +``` diff --git a/lsp-proxy/README.md b/lsp-proxy/README.md new file mode 100644 index 0000000..bf0c3eb --- /dev/null +++ b/lsp-proxy/README.md @@ -0,0 +1,107 @@ +# LSP Proxy + +The LSP proxy bridges browser WebSocket messages from CodeMirror to +`umple-lsp-server`, which speaks the Language Server Protocol over stdio. + +Browsers cannot spawn or talk to stdio language servers directly, so this +service owns the transport boundary and process lifecycle. + +## Runtime Shape + +``` +CodeMirror LSP client + | + | WebSocket JSON-RPC at /ws/lsp?session={modelId} + v +lsp-proxy + | + | Content-Length framed JSON-RPC over stdio + v +umple-lsp-server --stdio +``` + +In development, Vite rewrites `/ws/lsp` to the proxy's root WebSocket endpoint. +In production, frontend nginx does the same rewrite before proxying to the +`lsp-proxy` container. + +## Internal Architecture + +`server.js` is intentionally small and direct: + +- Creates an HTTP server for `/health` and `/status`. +- Creates a WebSocket server on the same port. +- Validates `session` query parameters with `SESSION_ID_RE`. +- Resolves the model directory under `UMP_BASE_DIR` and rejects missing or + path-traversal attempts. +- Enforces a global process cap with `LSP_MAX_PROCESSES`. +- Spawns `LSP_COMMAND --stdio` for each accepted WebSocket connection. +- Sets `UMPLE_SESSION_DIR` for the child process so the LSP server can read the + model workspace. +- Optionally passes `UMPLESYNC_JAR_PATH` through to the LSP process. +- Converts WebSocket JSON messages into LSP stdio frames. +- Parses LSP stdout frames and sends JSON messages back to the browser. +- Kills the child process when the socket closes or the process exits. + +The status endpoint reports process counts, rejected connections, configured +command, base directory, process limit, debug mode, PID, port, and uptime. + +## Frontend Contract + +The frontend LSP client lives in: + +- `frontend/src/hooks/useLsp.ts` +- `frontend/src/codemirror/lsp.ts` + +The browser sends `session={modelId}`. The proxy requires the corresponding +directory to exist under `UMP_BASE_DIR`, which is the same mounted model store +used by the backend. The frontend keeps all tab files synchronized so the LSP +server can provide cross-file diagnostics, definitions, references, and rename +behavior. + +## How This Differs From The Original + +Legacy UmpleOnline also had a WebSocket-to-stdio LSP proxy under +`~/umple/umpleonline/scripts/lsp-proxy/`, configured from the PHP-rendered +page. In that stack, `umple.php` emitted globals such as +`window.UMPLE_LSP_WS_URL`, `window.UMPLE_LSP_TOKEN`, and `window.UMPLE_UMP_BASE`. +The legacy proxy validated HMAC tokens from PHP and was run under the legacy +nginx/PHP/supervisord container setup. + +This rewrite keeps the same core transport idea but moves it into the modern +service architecture: + +- The proxy is a top-level Docker Compose service instead of a script inside + the PHP application tree. +- The browser reaches it through `/ws/lsp`, the same-origin route owned by the + frontend dev/prod proxy layer. +- The current contract validates the model directory and session ID, but does + not depend on PHP-generated LSP tokens. +- Runtime observability is exposed through `/health` and `/status` for Compose + and the backend status page. +- Model files live under the shared `/data/models` mount used by the Go + backend, not the legacy `/var/www/ump` tree. + +## Development + +From this folder: + +```bash +npm install +npm start +``` + +Normally this service is run by Docker Compose because the image installs +`umple-lsp-server` and mounts the model store: + +```bash +make dev-backend +``` + +Useful environment variables: + +- `LSP_PORT` - WebSocket/HTTP port, default `9999`. +- `UMP_BASE_DIR` - model directory root, default `/data/models`. +- `LSP_COMMAND` - language server command, default `umple-lsp-server`. +- `UMPLESYNC_JAR_PATH` - optional path passed through to the LSP server. +- `LSP_MAX_PROCESSES` - global child-process limit, default `20`. +- `LSP_DEBUG=1` - logs LSP method names as messages flow through the proxy.