docs(ops): FDD-OPS-019 — investigate Jira JQL pagination cap (BG short-fetch)#9
Merged
nascimentolimaandre-cloud merged 1 commit intomainfrom Apr 30, 2026
Conversation
…t-fetch)
OBSERVAÇÃO
Backfill comprehensive de 2026-04-30 (reset watermark issues → 2020-01-01)
processou apenas ~64% das issues do projeto BG (126k de 197.760), mesmo:
- tracker.finish('done') chamado normalmente (sem error)
- last_error NULL no pipeline_progress
- approximate-count consistente em 197.762
- coverage no DB confirma 197.760 BG issues existem
Iterator fetch_issues_batched terminou no caminho normal: nextPageToken
retornou vazio ou response.issues=[] após 126k. Não foi crash, timeout
nem retry esgotado.
HIPÓTESES (a investigar)
1. Hard cap interno da Jira em pagination cursor com ORDER BY updated
em busca muito antiga (since=2020 = ~5 anos)
2. Limit não-documentado em /rest/api/3/search/jql (novo endpoint)
3. approximate-count over-conta archived/deleted issues
4. Race condition em refresh de índice durante sweep
WORKAROUND PROPOSTO
Date-chunking quando pre-flight count > 100k:
jql = 'project = X AND updated >= START AND updated < END'
para chunks de ~2 anos. Já compatível com per-scope watermarks
(FDD-OPS-014) e per-scope progress (FDD-OPS-015).
IMPACTO ATUAL (não bloqueia R1)
Coverage tenant-wide ficou em 54% pós-backfill (era 13% pré-fix).
Os ~72k BG issues "missing" mantêm dados antigos mas:
- Métricas para os 240k issues frescos funcionam corretamente
- Incremental sync corrige naturalmente quando issues são updated
- Aceitável para PoC + Webmotors demos
DECISÃO
Aceitar 54% e abrir investigação como FDD-OPS-019 P1 (S, 1 dia).
Crítico antes de enterprise tenants com projetos >100k.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documenta achado durante backfill comprehensive de 2026-04-30: projeto BG terminou com
status='done'mas processou apenas ~64% das issues (126k de 197.760), sem error/crash/timeout. Hipótese principal: hard cap interno da Jira em cursor pagination comORDER BY updatedem buscas muito antigas (since=2020).Por que abrir agora (ao invés de fix imediato)
Coverage atual após backfill é 54% tenant-wide (era 13% pré-fix). Métricas funcionam corretamente para os 240k issues frescos. Os ~72k BG issues "missing" mantém dados antigos mas:
Crítico antes de enterprise tenants com projetos > 100k.
Investigation plan (proposto, S = 1 dia)
fetch_issues_batched, capturar resposta crua quandonextPageTokenvai vazioupdated >= 2020 AND < 2022, etc.). Compatível com per-scope watermarks (FDD-OPS-014) e per-scope progress (FDD-OPS-015) já shippadas.items_done < items_estimate × 0.9e cria event nopipeline_eventsAcceptance criteria
Stats
🤖 Generated with Claude Code