From 57119e5e331afa4f6dc8d56ae49b276f8570e82c Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Fri, 9 Jan 2026 18:54:53 -0600 Subject: [PATCH 1/7] Enhance README for Orchestrator module with detailed scope, definitions, interface and Input/Outputs. --- modules/orchestrator/README.md | 105 ++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 2 deletions(-) diff --git a/modules/orchestrator/README.md b/modules/orchestrator/README.md index 601f6de..1eb560b 100644 --- a/modules/orchestrator/README.md +++ b/modules/orchestrator/README.md @@ -1,4 +1,105 @@ # Orchestrator -Owner: Adrián Con -Goal: /plan_and_execute + reproducible runs + traces/logging. +- **Owner:** Adrián Con García +- **Module Path:** `modules/orchestrator` +- **Goal:** Provide a central orchestrator service that calls OpenPolicyStack microservices and produces reproducible “runs” with basic logs/traces. +--- + +## 1) Scope (What This Module Does) + +The **Orchestrator** coordinates end-to-end workflows across OpenPolicyStack modules. + +It is responsible for: +- Receiving a **Scenario Request** (what to run + which modules + parameters). +- Creating a **Reproducible Run Record** (e.g: Inputs, timestamps, versions, outputs). +- Executing a workflow via a single entrypoint: **`/plan_and_execute`**. +- Calling module microservices (HTTP) and collecting their outputs/artifacts. +- Writing basic **Logs** and optional **Trace Data** for transparency/debugging. +- Returning a structured response that can be plugged into a demo portal/workflow. + +**Non-Goals (MVP / Scope NOW)** +- Not an API Gateway or Portal (handled by the **Integration UI / API Gateway** effort). +- Not advanced **Monitoring & Telemetry** (phase 2). +- Not **Privacy & PII Redaction** (phase 2 / optional later). +- Not complex workflow engines (queues, distributed scheduling, etc.); keep sequential execution. +- Not implementing the analytics logic of other modules. +Out of scope for NOW; may be integrated later as separate services. + +--- + +## 2) Definitions (plain language) +- **Run:** One execution of the system for a given scenario + parameters. +- **Run ID:** Unique identifier for a run (used to find outputs/logs later). +- **Plan:** A list of steps the orchestrator will execute (e.g: call policy simulator → call strategy agent). +- **Artifact:** A saved output file (plot image, brief markdown, JSON results, etc.). +- **Trace/Logs:** A record of what happened during a run (steps, timing, success/failure). + +--- + +## 3) Interface (What This Module Exposes) + +### MVP API (Proposed Interface) +The orchestrator exposes a small HTTP API. + +#### `POST /plan_and_execute` +Create a run, generate a simple plan (sequence of module calls), execute it, and persist inputs/outputs/logs under a `run_id`. + +**Request (example):** This request starts a new orchestrated run for a demo policy scenario. It specifies which modules to execute and provides the scenario inputs (e.g: country, time horizon, and policy parameters like a VAT rate change). +```json +{ + "scenario_id": "demo-scenario-001", + "modules": ["policy-simulator", "strategy-agent"], + "inputs": { + "country": "DR", + "time_horizon_years": 5, + "policy_parameters": { + "vat_rate": 0.18 + } + }, + "run_options": { + "seed": 42, + "save_artifacts": true + } +} +``` + + +**Response (example):** The orchestrator returns a unique `run_id`, the execution plan it followed, and pointers to the saved outputs (KPIs/plots/brief) plus where logs and artifacts were stored for reproducibility. + +```json +{ + "run_id": "run_2026-01-09T12-34-56Z_ab12cd", + "status": "completed", + "plan": [ + {"step": 1, "module": "policy-simulator", "action": "execute"}, + {"step": 2, "module": "strategy-agent", "action": "execute"} + ], + "results": { + "policy-simulator": {"kpis": {"gdp_growth": 0.02}, "artifacts": ["kpis.json", "plot.png"]}, + "strategy-agent": {"brief": "brief.md", "artifacts": ["brief.md"]} + }, + "artifacts_path": "runs/run_2026-01-09T12-34-56Z_ab12cd/", + "logs_path": "runs/run_2026-01-09T12-34-56Z_ab12cd/logs.jsonl" +} +``` +`GET /runs/{run_id}` + +Returns the saved run metadata (inputs, plan, results pointers). + +`GET /health` + +Simple healthcheck endpoint. + +## 4) Inputs → Outputs (MVP) + +### Inputs +- `scenario_id` (string) +- `modules` (list of module names to call) +- `inputs` (JSON payload passed to modules) +- optional `run_options` (seed, toggles for saving artifacts, etc.) + +### Outputs +- `run_id` +- `plan` (what steps were executed) +- per-module results (JSON + artifact references) +- paths/pointers to stored logs and artifacts for reproducibility \ No newline at end of file From 265fe7a05abf9cad85f5a5dbe8b67c70e050f30f Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Sat, 10 Jan 2026 12:14:25 -0600 Subject: [PATCH 2/7] Added sections for run storage and assumptions to call modules to the README. --- modules/orchestrator/README.md | 44 +++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/modules/orchestrator/README.md b/modules/orchestrator/README.md index 1eb560b..a85c977 100644 --- a/modules/orchestrator/README.md +++ b/modules/orchestrator/README.md @@ -102,4 +102,46 @@ Simple healthcheck endpoint. - `run_id` - `plan` (what steps were executed) - per-module results (JSON + artifact references) -- paths/pointers to stored logs and artifacts for reproducibility \ No newline at end of file +- paths/pointers to stored logs and artifacts for reproducibility + +## 5) Run Storage (Reproducibility) + +Every call to `POST /plan_and_execute` creates a **run**. +A run is a single execution of a scenario with specific modules + inputs. + +To make runs **reproducible and traceable**, the orchestrator saves: +- The original request (inputs, chosen modules, options like seed), +- What steps were executed (the “plan”), +- Each module’s returned results, +- Basic logs of what happened. + +This is stored under a unique `run_id` in a folder like: + + +
runs/<run_id>/
+  run.json        # input request + derived plan + timestamps
+  results.json    # merged results (pointers to artifacts)
+  logs.jsonl      # structured logs (one JSON per line)
+  artifacts/
+    <module-name>/
+      ...         # plots, JSONs, briefs, etc.
+ +The API response includes the `run_id` and (optionally) paths/pointers to the saved logs and artifacts. + +## 6) How modules are called (assumptions for integration) + +For MVP integration, the orchestrator assumes modules are reachable over **HTTP** (Docker-first) and expose a minimal interface so they can be called consistently. + +The proposed minimal module contract is documented in the repo Issue: +**“MVP module interface contract (proposed)”** (see GitHub Issues). + +In short, the orchestrator expects: +- `GET /health` for basic service readiness checks +- One primary execution endpoint (preferred: `POST /execute`, or a module-specific endpoint such as `/score`, `/risk`, `/run_scenario`) +- JSON-in / JSON-out +- A consistent response “envelope” including: + - `status`, `module`, `outputs` + - Optional `artifacts` and `evidence` fields + +> Note: Endpoint naming and exact payload fields may be refined during integration month. The orchestrator will prioritize compatibility with the agreed contract in the Issue and adapt via lightweight adapters if needed. + From 2c2e00281d739cb2eb5cd9de2f051a939d3907b8 Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Wed, 28 Jan 2026 10:56:17 +0100 Subject: [PATCH 3/7] Initial commit --- modules/orchestrator/README.md | 43 ++++++++++++++++++++++++++++++++-- 1 file changed, 41 insertions(+), 2 deletions(-) diff --git a/modules/orchestrator/README.md b/modules/orchestrator/README.md index a85c977..5469c19 100644 --- a/modules/orchestrator/README.md +++ b/modules/orchestrator/README.md @@ -27,7 +27,7 @@ Out of scope for NOW; may be integrated later as separate services. --- -## 2) Definitions (plain language) +## 2) Definitions (Plain Language) - **Run:** One execution of the system for a given scenario + parameters. - **Run ID:** Unique identifier for a run (used to find outputs/logs later). - **Plan:** A list of steps the orchestrator will execute (e.g: call policy simulator → call strategy agent). @@ -128,7 +128,7 @@ This is stored under a unique `run_id` in a folder like: The API response includes the `run_id` and (optionally) paths/pointers to the saved logs and artifacts. -## 6) How modules are called (assumptions for integration) +## 6) How Modules Are Called (Assumptions for Integration) For MVP integration, the orchestrator assumes modules are reachable over **HTTP** (Docker-first) and expose a minimal interface so they can be called consistently. @@ -145,3 +145,42 @@ In short, the orchestrator expects: > Note: Endpoint naming and exact payload fields may be refined during integration month. The orchestrator will prioritize compatibility with the agreed contract in the Issue and adapt via lightweight adapters if needed. +## 7) How To Run (Local / Docker) + +> Status: This module is currently in setup phase. The commands below describe the intended MVP run method and will be made runnable as the skeleton is implemented. + +### Docker (recommended for reproducibility) +Planned workflow: +```bash +# from modules/orchestrator +docker build -t openpolicystack-orchestrator:dev . +docker run --rm -p 8000:8000 \ + -v $(pwd)/runs:/app/runs \ + openpolicystack-orchestrator:dev +``` + +### Local (for development) +Planned workflow: +```bash +# from modules/orchestrator +pip install -r requirements.txt +# example server command (framework to be confirmed) +python -m src.main +``` +## 8) Quickstart Demo (Planned) + +> Status: This is the intended MVP smoke test once the orchestrator skeleton is implemented. + +1) Start the orchestrator (see Section 7) +2) Start at least one module service (or a stub) reachable by the orchestrator +3) Trigger a run: + +```bash +curl -X POST http://localhost:8000/plan_and_execute \ + -H "Content-Type: application/json" \ + -d @examples/plan_and_execute.request.json +``` +Expected behavior (once implemented): +- Returns a JSON response containing a `run_id` +- Creates `runs//` with `run.json`, `results.json`, `logs.jsonl`, and `artifacts/` + From ab9dacc2e8a458bb0e222a27a0271ebc220c9c74 Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Wed, 28 Jan 2026 11:16:48 +0100 Subject: [PATCH 4/7] Kept only 1 PR template --- .github/PULL_REQUEST_TEMPLATE.md | 7 ------- 1 file changed, 7 deletions(-) delete mode 100644 .github/PULL_REQUEST_TEMPLATE.md diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md deleted file mode 100644 index 7d8291d..0000000 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ /dev/null @@ -1,7 +0,0 @@ -## Summary -Explain what changed and why. - -## Checklist -- [ ] No secrets/credentials committed -- [ ] Docs updated (if needed) -- [ ] Tests added/updated (if applicable) From be8f2349fce252bf6cffd1b60dc57b064e18a1cf Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Wed, 28 Jan 2026 14:48:15 +0100 Subject: [PATCH 5/7] Nothing to commit. --- .github/pull_request_template.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 2bd66b3..7d8291d 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,10 +1,7 @@ -## What does this PR change? -- - -## Which module? -- +## Summary +Explain what changed and why. ## Checklist -- [ ] README updated (how to run + demo) -- [ ] Inputs/outputs documented -- [ ] No credentials / private data +- [ ] No secrets/credentials committed +- [ ] Docs updated (if needed) +- [ ] Tests added/updated (if applicable) From 3ba62e9bbaca5b4deec73931f9ec35b8571956db Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Mon, 2 Feb 2026 19:07:57 +0100 Subject: [PATCH 6/7] Test change for PR Approval. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ba68d89..404890c 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # OpenPolicyStack -Open-source modular stack for AI-enhanced policy making. Includes reusable microservices for data ingestion, simulation, analytics, dashboards, strategy agents, and decision-support tools. +Open-source modular stack for AI-enhanced policy making. Includes reusable microservices for data ingestion, simulation, analytics, dashboards, strategy agents, and decision-support tools. ## Modules - **Monitor** (`modules/monitor`): dashboards & reporting (Metabase), ETL jobs, telemetry. From 97b7d1122efa016a040bcd9a32f2e46fbee3de23 Mon Sep 17 00:00:00 2001 From: Adrian Con Date: Mon, 2 Feb 2026 19:08:39 +0100 Subject: [PATCH 7/7] Nothing to commit. --- .github/pull_request_template.md | 7 ------- 1 file changed, 7 deletions(-) delete mode 100644 .github/pull_request_template.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md deleted file mode 100644 index 7d8291d..0000000 --- a/.github/pull_request_template.md +++ /dev/null @@ -1,7 +0,0 @@ -## Summary -Explain what changed and why. - -## Checklist -- [ ] No secrets/credentials committed -- [ ] Docs updated (if needed) -- [ ] Tests added/updated (if applicable)