Skip to content

Add /statusz endpoint to expose re-indexing state#1085

Open
rhyswdev wants to merge 1 commit intoopensanctions:mainfrom
rhyswdev:add-statusz-endpoint
Open

Add /statusz endpoint to expose re-indexing state#1085
rhyswdev wants to merge 1 commit intoopensanctions:mainfrom
rhyswdev:add-statusz-endpoint

Conversation

@rhyswdev
Copy link
Copy Markdown

@rhyswdev rhyswdev commented Apr 1, 2026

Closes #1086

Problem

/readyz returns 200 OK throughout an active re-index because yente continues serving queries from the old index while building the new one. This means there is no HTTP endpoint to determine whether a re-index (triggered by /updatez, cron, or startup) is still in progress or has completed.

Why existing endpoints don't solve this

Endpoint Behaviour during re-index Why it's insufficient
GET /readyz Returns 200 OK Old index is healthy, so health check passes
GET /catalog index_stale / index_current reflect the catalog version vs indexed version — not whether indexing is running No state change if upstream data version hasn't changed (e.g. forced re-index); can't distinguish "hasn't started" from "in progress" from "finished"
POST /updatez?sync=true Blocks until done if it acquires the lock If another process already holds the lock, it returns 200 ok immediately (indexer.py:334-337) — a silent false "done" signal

Current workaround

Poll the lock document directly in the search backend:

GET {search_url}/{index_name}-locks/_doc/lock

This works but has drawbacks:

  • Couples consumers to yente's internal search backend implementation
  • The found field alone is insufficient — the lock has a 10-minute expiry (LOCK_EXPIRATION_TIME), so a stale lock from a crashed indexer shows found: true but isn't active. Callers must also check _source.acquired_at is within the expiry window.

Solution

Add a GET /statusz endpoint that checks the distributed indexing lock and returns:

  • {"status": "ok"} — no re-index in progress
  • {"status": "indexing"} — re-index is in progress

This follows the existing *z naming convention (healthz, readyz, updatez) and reuses the StatusResponse model. The endpoint is read-only and has no side effects on the indexing process.

Changes

  • yente/search/lock.py — Add check_is_locked(): read-only check for an active (non-expired) lock, handles missing lock index gracefully
  • yente/routers/admin.py — Add GET /statusz endpoint in the "System information" tag
  • tests/test_base.py — Add test_statusz() following the existing test pattern
  • docs/deploy/monitoring.md — Document the new endpoint alongside /healthz and /readyz

🤖 Generated with Claude Code

The existing /readyz endpoint always returns "ok" during re-indexing
because the old index continues to serve traffic. This makes it
impossible for downstream systems to know when a re-index completes.

The new /statusz endpoint checks the distributed indexing lock and
returns {"status": "indexing"} while a re-index is in progress, or
{"status": "ok"} otherwise.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@leonhandreke
Copy link
Copy Markdown
Contributor

Hi, thanks for taking the time to create this. Just out of interest, are you or would you be actively using this in your deployment of yente? I can see what it does, but I'm a bit wary of adding a new endpoint before figuring out what exactly the requirements are. What we've cared about so far is index staleness (as in, how long has it been since it's been updated), monitoring the time that it takes to index (which is usually just a few seconds for deltas) has not been a requirement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No HTTP endpoint to check if re-indexing is in progress

2 participants