Define Input and Output Schema for Financial Data APIs

[pouria-mirebrahimi]

Story MP-018: Define Input and Output Schema for Financial Data APIs

Description:
Organize and conduct a series of meetings with stakeholders, data engineers, and analysts to define a clear, standardized schema for all API inputs and outputs. This ensures consistency across Market Prices, News Sentiment, and Fundamentals data and smooth integration with downstream systems.

Tasks:

• Planning & Scheduling

  • Identify key participants (data engineers, analysts, product owners)
  • Schedule recurring meetings to discuss schema requirements
  • Prepare initial agenda and reference materials (current datasets, expected use cases)

• Requirements Gathering

  • Review existing data sources (YFinance, AlphaVantage, Finnhub, FMP, internal datasets)
  • Identify mandatory vs optional fields for each data type
  • Discuss expected data types, formats, and units
  • Consider pagination, filtering, and metadata needs

• Schema Design Workshops

  • Draft preliminary schema for each API:

    • Market Prices: {symbol, timestamp, open, high, low, close, volume}
    • News Sentiment: {symbol, date, source, headline, sentiment_score}
    • Fundamentals: {symbol, date, metric_name, value}

  • Validate schema against real data examples

  • Incorporate feedback from stakeholders

• Documentation & Approval

  • Consolidate final schema definitions into a shared document
  • Record decisions, edge cases, and conventions
  • Obtain approval from key stakeholders before implementation

Acceptance Criteria:

• Schema defined for Market Prices, News Sentiment, and Fundamentals APIs
• All fields, data types, and formats are agreed upon and documented
• Schema document is shared and approved by stakeholders
• Any changes to schema follow a documented review process

Estimation: 2–3 days (including meetings and follow-up documentation)
Dependencies: -
Status: Planned

[pouria-mirebrahimi]

Work collaboratively on this task, please;

There are two primary schemas, one for our private use according to the internal database/API, and the other one including YFinance, Finnhub, etc..., for documenting in the repository.

@Farhad-Valipour @nikbakhtenb4