Add tasks

.github/workflows/pages.yml

@@ -0,0 +1,40 @@
+# Deploy static site from docs/ to GitHub Pages (configure repo: Settings → Pages → Source: GitHub Actions)
+name: Deploy GitHub Pages
+
+on:
+  push:
+    branches:
+      - main
+      - dev
+      - feat/landing-page
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Configure Pages
+        uses: actions/configure-pages@v4
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: docs
+
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

README.md

@@ -126,7 +126,11 @@ For chatbot integration, copy `data360://system-prompt` into your system prompt.
 
 ## Documentation
 
-Full documentation is available at: **https://worldbank.github.io/data360-mcp**
+**Project site:** [worldbank.github.io/data360-mcp](https://worldbank.github.io/data360-mcp) — landing page with features, tools, and connection details.
+
+A markdown overview lives in [docs/overview.md](docs/overview.md). The site is deployed with [GitHub Actions](.github/workflows/pages.yml) on pushes to `main` or `dev`. In the repository **Settings → Pages**, set **Build and deployment** source to **GitHub Actions** (first-time setup).
+
+**Preview locally:** from the repository root, run `python -m http.server --directory docs` and open `http://127.0.0.1:8000/`.
 
 For developer setup, testing, and contribution instructions, see [DEVELOPMENT.md](DEVELOPMENT.md).
 

TODO/CROSS-REPO-GRAPH.md

@@ -0,0 +1,3 @@
+# Cross-repo task dependencies
+
+See the canonical graph in **[data-ai-chatbot/TODO/CROSS-REPO-GRAPH.md](../../data-ai-chatbot/TODO/CROSS-REPO-GRAPH.md)**.

TODO/MCP-001-system-prompt-sync.md

@@ -0,0 +1,40 @@
+---
+id: MCP-001
+repo: data360-mcp
+title: SYSTEM_PROMPT — align with chatbot Writer/Planner rules
+status: pending
+priority: medium
+depends_on:
+  - BE-001
+blocks: []
+external_ref: vercel-ai-chatbot/TODO/BE-001-writer-planner-prompts.md
+---
+
+# MCP-001 — Sync `data360://system-prompt` with chatbot prompts
+
+## Goal
+
+Reduce drift between MCP-only LLM clients and the Data AI Chatbot: update `SYSTEM_PROMPT` in `src/data360/mcp_server/prompts.py` so tool-loop behavior and **final answer structure / link expectations** stay consistent with `vercel-ai-chatbot` `backend/app/ai/prompts.py` after **BE-001**.
+
+## Context
+
+- Resource registration: `src/data360/mcp_server/resources.py` exposes `data360://system-prompt`.
+- Do not duplicate entire Writer prompt if inappropriate for MCP; align **non-conflicting** bullets: section order hints, link discipline, when to call `data360_get_viz_spec`.
+
+## Implementation hints
+- **Entry point:** `src/data360/mcp_server/prompts.py` — `SYSTEM_PROMPT` string literal lines 11–56.
+- **Current structure:** 4 sections — `### Non-negotiable rule` (line 15), `### Operating loop` (line 19, 5-step workflow: search → codelist → availability → get data → visualize), `### Defaults` (line 47, 20-year range hardcoded in text), `### Output behavior` (line 52).
+- **Exposed via:** `src/data360/mcp_server/resources.py` line 159–162 as MCP resource `data360://system-prompt`.
+- **Desired behavior:** After BE-001 finalises section order and link discipline in the chatbot Writer prompt, mirror the non-conflicting bullets here — specifically: section order hints for final answer, link discipline for API/indicator URLs, when to call `data360_get_viz_spec`. Do NOT duplicate the full Writer prompt.
+- **Test file:** No tests for SYSTEM_PROMPT content. After update, manually verify that `data360://system-prompt` resource returns the updated string via MCP inspector or a quick test script.
+- **Gotchas:** SYSTEM_PROMPT is a plain string — preserve existing markdown formatting exactly. The 20-year default is hardcoded in text (line 48–49), not a config variable. Sync changes in `prompts.py` to `README.md` and `docs/overview.md` if they quote the resource content.
+
+## Acceptance criteria
+
+- [ ] `SYSTEM_PROMPT` updated to reflect agreed structure and link/viz nudges (scoped to MCP tool use).
+- [ ] README/docs that quote the resource updated if needed (`README.md`, `docs/overview.md`).
+- [ ] No contradiction with existing mandatory tool-call rules in the same file.
+
+## Dependencies
+
+- **BE-001** (vercel-ai-chatbot) — source of truth for product wording; complete or sync in pair review.

TODO/MCP-002-vega-line-tooltips.md

@@ -0,0 +1,43 @@
+---
+id: MCP-002
+repo: data360-mcp
+title: Vega-Lite — default tooltips for line/area charts
+status: pending
+priority: medium
+depends_on: []
+blocks: []
+---
+
+# MCP-002 — Default chart tooltips
+
+## Goal
+
+Line (and similar) charts emitted by `get_viz_spec` show hover tooltips by default for readability in embedded previews (e.g. chat `ChartPreview`).
+
+## Context
+
+- Implementation likely in `src/data360/visualization.py` and/or `src/data360/viz_config.py` where Vega-Lite spec is built.
+- Frontend: `vercel-ai-chatbot` renders spec via vega-embed in `frontend/components/data360/chart-preview.tsx` — no change required if spec includes tooltips.
+
+## Implementation hints
+- **Entry point (Draco path):** `src/data360/visualization.py` line 555–556 — `tooltip_cols = list(viz_data.columns); chart = chart.encode(tooltip=tooltip_cols)`. This runs after Draco rendering.
+- **Entry point (fallback path):** `src/data360/visualization.py` line 603 — `base = alt.Chart(viz_data).mark_line().encode(tooltip=fc)`. Fallback uses `fc` (field config) for tooltip.
+- **Current behavior:** Tooltip is a flat list of all columns. Column names at this point are already renamed: `time_period → year`, `obs_value → value`, `ref_area → country` (renaming happens at lines 414–434). Tooltip shows raw renamed columns with no formatting.
+- **Desired behavior:** Structured tooltips per chart type — for line/area: show `year`, `value`, `country` with human-readable labels and number formatting. Use `alt.Tooltip(field, title, format)` objects instead of raw column names.
+- **Chart types to cover:** line (line 149), area (line 167) as minimum. bar (line 155), point (line 161), tick (line 173) as stretch.
+- **Test file:** `tests/test_visualization.py` — existing class `TestGetVizSpecDracoFallbackWarning` (lines 11–147). Add a new test class `TestTooltipEncoding` that asserts tooltip fields and titles on the returned spec dict.
+- **Gotchas:** Tooltip encoding happens AFTER Draco (line 555) — this is intentional to allow override. Fallback path (line 603) hardcodes `mark_line()` regardless of chart type — tooltip fix should handle both paths. Codelist mapping runs before viz (lines 421–431) so `country` column contains names not codes at tooltip time.
+
+## Acceptance criteria
+
+- [ ] Line/area (and other relevant marks) include `tooltip` encoding or mark-level tooltip defaults.
+- [ ] Tests in `tests/test_visualization.py` updated or added to assert tooltip presence in spec JSON.
+- [ ] Performance/tooltip density acceptable for many points (coordinate with MCP-003).
+
+## Dependencies
+
+- None.
+
+## Related
+
+- **MCP-003** often touches the same spec-building code; coordinate or sequence to reduce merge conflicts (soft dependency only).

TODO/MCP-003-beeswarm-multi-country.md

@@ -0,0 +1,43 @@
+---
+id: MCP-003
+repo: data360-mcp
+title: Many countries — beeswarm/strip or facet instead of clutter
+status: pending
+priority: medium
+depends_on: []
+blocks: []
+soft_depends_on:
+  - MCP-002
+---
+
+# MCP-003 — High cardinality series (multi-country) visualization
+
+## Goal
+
+When many countries/series make line charts unreadable or data is truncated, prefer a strip/beeswarm-style layout, faceting, or another documented strategy in the viz pipeline.
+
+## Context
+
+- `src/data360/viz_config.py`, `src/data360/visualization.py`.
+- May require `data360_get_supported_chart_types` updates if new chart type names are exposed.
+- Product threshold (country count) may come from design — document default in code comments.
+
+## Implementation hints
+- **Cardinality detection:** `src/data360/visualization.py` lines 490–515 — loops over breakdown dims (`country`, `sex`, `age`, `urbanisation`), picks first with `.nunique() > 1` as color dim. No upper-bound check for "too many series".
+- **Chart type selection:** `src/data360/viz_config.py` → `should_use_temporal_x_axis()` lines 367–448. Returns `(True, None)` for time-series or `(False, categorical_field)` for categorical. Preference scores by chart type: tick=1.0, point=0.7, bar=0.5, line=0.2, area=0.1.
+- **Current behavior:** With 20+ countries, Draco picks a line or bar chart with color encoding for each country. Above ~10 series, the chart becomes unreadable — overlapping lines, crowded legend.
+- **Desired behavior:** When `country.nunique() > N` (suggested threshold: 10, make it configurable) AND single-year data (temporal cardinality = 1), switch to strip/beeswarm: `mark_point()` with jitter, value on y-axis, country as color limited to top-N or omitted.
+- **Beeswarm in Vega-Lite:** No native beeswarm mark. Use `mark_point()` with `transform: [{"calculate": "random()", "as": "jitter"}]` and encode jitter on x-axis. Or use Altair's `mark_tick()` (already in the chart type list at line 173) as a simpler strip chart alternative.
+- **Where to add:** Add `should_use_beeswarm(viz_data)` function in `viz_config.py` near `should_use_temporal_x_axis()`. Call it in `visualization.py` before Draco encoding (around line 490) when cardinality exceeds threshold.
+- **Test file:** `tests/test_visualization.py`. Add `TestHighCardinalityChartSelection` — assert beeswarm/strip selected when country count > threshold, line chart when ≤ threshold.
+- **Gotchas:** Vega-Lite jitter requires a `calculate` transform — adds complexity to the spec. `tick` mark already exists and may be sufficient as a simpler strip chart without jitter. Clarify with product whether jitter is required or if a tick/strip is acceptable. Cardinality thresholds should be centralised (add to `MCPServerSettings` in `config.py` alongside future `MCP_CACHE_TTL`).
+
+## Acceptance criteria
+
+- [ ] Heuristic selects alternate mark/layout above a configurable threshold.
+- [ ] Unit tests cover threshold boundary and spec shape.
+- [ ] Planner/chatbot prompts (BE-001) mention behavior if user-facing copy needed — optional follow-up.
+
+## Dependencies
+
+- **MCP-002** (soft): tooltip behavior should remain sane on the chosen mark.

TODO/MCP-004-viz-empty-data-guard.md

@@ -0,0 +1,47 @@
+---
+id: MCP-004
+repo: data360-mcp
+title: viz — guard against empty data in generate_vega_spec()
+status: pending
+depends_on: []
+blocks: []
+source: pr-review
+file_ref: src/data360/mcp_server/viz.py
+line_ref: 142
+---
+
+# MCP-004 — Handle empty `data` list in `generate_vega_spec()`
+
+## Goal
+
+Prevent `generate_vega_spec()` from crashing with a `KeyError` when `data` is an empty list. Return a user-friendly error message or a graceful empty chart spec instead.
+
+## Context
+
+- PR review flagged that `data[0]['date']` raises `KeyError` when `data=[]`.
+- The referenced file path in the review is `src/data360/mcp_server/viz.py` (line 142); the actual implementation lives in `src/data360/visualization.py` and/or `src/data360/viz_config.py` — confirm exact location before patching.
+- Empty data can legitimately occur when an API query returns no results for a given filter combination; callers should receive a clear signal rather than an unhandled exception.
+
+## Acceptance criteria
+
+- [ ] `generate_vega_spec()` (or equivalent entry point) checks for an empty `data` list before accessing `data[0]`.
+- [ ] Returns either a well-formed error dict/message or a minimal empty-chart Vega-Lite spec (document the chosen convention in a code comment).
+- [ ] Unit test added: `test_generate_vega_spec_empty_data` asserts no exception is raised and the return value is well-formed.
+- [ ] No regression on existing non-empty data tests.
+
+## Implementation hints
+
+- **Entry point:** `src/data360/visualization.py` → `get_viz_spec()` (function starting ~line 183). The `_fetch_data_internal()` helper at line ~109 already raises `ValueError("No data found...")` for empty API responses, and `get_viz_spec()` catches that at line ~243. However, if `data` reaches downstream spec-building steps (e.g. after filtering in the `relevant_fields` branch), an empty DataFrame/list can still cause `KeyError` or `IndexError` on first-row access.
+- **Current behavior:** When `data` is empty after filtering, the code may attempt `data[0]['date']` (or an equivalent first-row access) and raise `KeyError` uncaught.
+- **Desired behavior:** Add a guard immediately after any potential reduction to empty (e.g. after `viz_data = data[valid_cols].copy()`) that returns `err("Error: No data available after applying the requested filters.")`. The existing check at line ~439 (`if viz_data.empty: return err(...)`) provides the pattern to follow.
+- **Test file:** `tests/test_visualization.py` — add a test case in the existing `TestGetVizSpecDracoFallbackWarning` class or a new class. Mock `_fetch_data_internal` to return a DataFrame that becomes empty after field filtering, and assert the returned dict has `url=None` and a non-None `error`.
+- **Prior art:** `_fetch_data_internal()` at line ~109 already checks `if not raw_data: raise ValueError(...)` — follow the same early-exit pattern. The `if viz_data.empty: return err(...)` guard at line ~439 shows how to do it after cleaning.
+- **Gotchas:** The file path in the original PR review (`src/data360/mcp_server/viz.py`) does not exist yet. If a separate `viz.py` module is created as part of a refactor, place the guard there; otherwise add it to `src/data360/visualization.py`.
+
+## Dependencies
+
+- None.
+
+## Related
+
+- **MCP-002** / **MCP-003** — touch the same spec-building pipeline; sequence or coordinate to avoid merge conflicts.

TODO/MCP-005-cache-configurable-ttl.md

@@ -0,0 +1,44 @@
+---
+id: MCP-005
+repo: data360-mcp
+title: cache — make TTL configurable via MCP_CACHE_TTL env var
+status: pending
+depends_on: []
+blocks: []
+source: pr-review
+file_ref: src/data360/mcp_server/cache.py
+line_ref: 23
+---
+
+# MCP-005 — Configurable cache TTL via `MCP_CACHE_TTL` environment variable
+
+## Goal
+
+Replace the hardcoded 300-second TTL in the cache module with a value read from the `MCP_CACHE_TTL` environment variable, keeping 300 as the default.
+
+## Context
+
+- PR review flagged a hardcoded `300` (seconds) in `src/data360/mcp_server/cache.py` line 23.
+- The file `src/data360/mcp_server/cache.py` does not currently exist in the repo; the cache logic may live elsewhere (e.g. `src/data360/config.py`). Locate the hardcoded value before patching.
+- Making the TTL configurable allows operators to tune cache aggressiveness in different deployment environments without code changes.
+
+## Acceptance criteria
+
+- [ ] Cache TTL is read from `os.environ.get("MCP_CACHE_TTL", 300)` (or equivalent via `src/data360/config.py` if that module centralises env vars).
+- [ ] Value is cast to `int` with a clear `ValueError` or fallback if the env var is set to a non-integer.
+- [ ] Default behaviour (TTL = 300 s) is unchanged when the env var is unset.
+- [ ] `MCP_CACHE_TTL` documented in `README.md` (or equivalent config docs) under environment variables.
+- [ ] Unit test added verifying TTL is picked up from the environment variable.
+
+## Implementation hints
+
+- **Entry point:** The file `src/data360/mcp_server/cache.py` (line 23 per the PR review) does not yet exist in the repo. The likely location for the hardcoded TTL is wherever a TTL-based caching layer is added. The existing settings system in `src/data360/config.py` uses `pydantic-settings` with `env_prefix="MCP_"` (`MCPServerSettings` class, line ~10). The cleanest approach is to add a `cache_ttl: int = Field(default=300, ...)` field to `MCPServerSettings` — with `env_prefix="MCP_"` already in place, it will automatically read `MCP_CACHE_TTL` from the environment.
+- **Current behavior:** TTL is hardcoded to `300` (seconds) in the cache module (or will be when that module is created).
+- **Desired behavior:** TTL is read from `MCP_CACHE_TTL` env var via `MCPServerSettings.cache_ttl`, defaulting to `300`. Cache module imports `get_mcp_server_settings()` and uses `settings.cache_ttl`.
+- **Test file:** No dedicated cache test file exists yet. Create `tests/test_cache.py`. Test that `MCPServerSettings(MCP_CACHE_TTL=600).cache_ttl == 600` and that the default is `300` when env var is unset.
+- **Prior art:** `src/data360/config.py` `MCPServerSettings` — all other MCP tunables (port, log_level, charts_api_url) follow exactly this pattern. Add `cache_ttl` here to stay consistent.
+- **Gotchas:** `get_mcp_server_settings()` is decorated with `@ft.cache` (functools LRU cache), meaning it returns the same instance after first call. In tests, ensure settings are created fresh with `MCPServerSettings(...)` directly rather than calling the cached factory, or use `monkeypatch.setenv` before the factory is first called.
+
+## Dependencies
+
+- None.

TODO/README.md

@@ -0,0 +1,34 @@
+# data360-mcp — task index
+
+Agent-oriented work items. Each task is a standalone `.md` file with context, acceptance criteria, and dependency metadata. (5 active)
+
+**Sibling repos:** [data-ai-chatbot/TODO](../data-ai-chatbot/TODO/README.md)
+
+## Task IDs
+
+| ID | File | Summary |
+|----|------|---------|
+| MCP-001 | [MCP-001-system-prompt-sync.md](./MCP-001-system-prompt-sync.md) | Reduce drift between MCP-only LLM clients and the Data AI Chatbot: update `SY... |
+| MCP-002 | [MCP-002-vega-line-tooltips.md](./MCP-002-vega-line-tooltips.md) | Line (and similar) charts emitted by `get_viz_spec` show hover tooltips by de... |
+| MCP-003 | [MCP-003-beeswarm-multi-country.md](./MCP-003-beeswarm-multi-country.md) | When many countries/series make line charts unreadable or data is truncated, ... |
+| MCP-004 | [MCP-004-viz-empty-data-guard.md](./MCP-004-viz-empty-data-guard.md) | Prevent `generate_vega_spec()` from crashing with a `KeyError` when `data` is... |
+| MCP-005 | [MCP-005-cache-configurable-ttl.md](./MCP-005-cache-configurable-ttl.md) | Replace the hardcoded 300-second TTL in the cache module with a value read fr... |
+
+## Dependency graph (this repo)
+
+```mermaid
+flowchart TB
+  MCP001["MCP-001 SYSTEM_PROMPT — align with cha"]
+  MCP002["MCP-002 Vega-Lite — default tooltips f"]
+  MCP003["MCP-003 Many countries — beeswarm/stri"]
+  MCP004["MCP-004 viz — guard against empty data"]
+  MCP005["MCP-005 cache — make TTL configurable "]
+  MCP002 -.-> MCP003
+```
+
+## How to use
+
+1. Point an agent at `TODO/` or a specific task file.
+2. Check `depends_on` in the task frontmatter before starting — all hard deps must be `done`.
+3. Claim a task by setting `status: in_progress` and committing immediately.
+4. After completing a task, set `status: done`, tick acceptance criteria checkboxes, and check `blocks` for newly unblocked tasks.

TODO/tasks.json

@@ -0,0 +1,8 @@
+{
+  "name": "data360-mcp",
+  "domains": ["MCP"],
+  "github_repo": "",
+  "peers": {
+    "data-ai-chatbot": "${DATA_AI_CHATBOT_PATH}"
+  }
+}