Skip to content

[M2][Epic 2.2] Add API polling and file upload source types #63

@POWDER-RANGER

Description

@POWDER-RANGER

Problem

Extend POST /api/sources to support API polling (simple GET requests) and CSV/JSON uploads.

Tasks

  • Add source type validation (api|rss|upload)
  • Implement API source: store endpoint URL, poll interval
  • Implement API source: HTTP client with error handling
  • Implement upload source: accept CSV/JSON via multipart form
  • Add data normalization layer (CSV → documents, JSON → documents)
  • Write integration tests for each source type
  • Update frontend to support all 3 types

Acceptance Criteria

  • POST /api/sources accepts RSS + API + Upload
  • ✅ Data normalized correctly (schema validated)
  • ✅ Ingestion works identically across all source types

Priority: P1 - High
Labels: backend, ingestion, M2, P1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions