Add /statusz endpoint to expose re-indexing state#1085
Open
rhyswdev wants to merge 1 commit intoopensanctions:mainfrom
Open
Add /statusz endpoint to expose re-indexing state#1085rhyswdev wants to merge 1 commit intoopensanctions:mainfrom
rhyswdev wants to merge 1 commit intoopensanctions:mainfrom
Conversation
The existing /readyz endpoint always returns "ok" during re-indexing
because the old index continues to serve traffic. This makes it
impossible for downstream systems to know when a re-index completes.
The new /statusz endpoint checks the distributed indexing lock and
returns {"status": "indexing"} while a re-index is in progress, or
{"status": "ok"} otherwise.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Hi, thanks for taking the time to create this. Just out of interest, are you or would you be actively using this in your deployment of yente? I can see what it does, but I'm a bit wary of adding a new endpoint before figuring out what exactly the requirements are. What we've cared about so far is index staleness (as in, how long has it been since it's been updated), monitoring the time that it takes to index (which is usually just a few seconds for deltas) has not been a requirement. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1086
Problem
/readyzreturns200 OKthroughout an active re-index because yente continues serving queries from the old index while building the new one. This means there is no HTTP endpoint to determine whether a re-index (triggered by/updatez, cron, or startup) is still in progress or has completed.Why existing endpoints don't solve this
GET /readyz200 OKGET /catalogindex_stale/index_currentreflect the catalog version vs indexed version — not whether indexing is runningPOST /updatez?sync=true200 okimmediately (indexer.py:334-337) — a silent false "done" signalCurrent workaround
Poll the lock document directly in the search backend:
This works but has drawbacks:
foundfield alone is insufficient — the lock has a 10-minute expiry (LOCK_EXPIRATION_TIME), so a stale lock from a crashed indexer showsfound: truebut isn't active. Callers must also check_source.acquired_atis within the expiry window.Solution
Add a
GET /statuszendpoint that checks the distributed indexing lock and returns:{"status": "ok"}— no re-index in progress{"status": "indexing"}— re-index is in progressThis follows the existing
*znaming convention (healthz,readyz,updatez) and reuses theStatusResponsemodel. The endpoint is read-only and has no side effects on the indexing process.Changes
yente/search/lock.py— Addcheck_is_locked(): read-only check for an active (non-expired) lock, handles missing lock index gracefullyyente/routers/admin.py— AddGET /statuszendpoint in the "System information" tagtests/test_base.py— Addtest_statusz()following the existing test patterndocs/deploy/monitoring.md— Document the new endpoint alongside/healthzand/readyz🤖 Generated with Claude Code