MongoDB News API

A high-performance REST API for accessing news articles from MongoDB with advanced filtering and cursor-based pagination.

🎯 Features

🔍 Advanced Filtering: Filter by date range, source, asset, and keywords
📄 Cursor Pagination: Efficient pagination for large datasets
🔐 API Key Auth: Secure access with API key authentication
🚀 High Performance: Async operations with Motor driver
📊 RESTful API: Standard HTTP methods and status codes
📖 Auto Documentation: Interactive API docs with Swagger UI
📊 Analytics & Aggregations: Get insights with aggregation endpoints
🛡️ Rate Limiting: 1000 requests per hour per API key (Default)
📝 Request Logging: Comprehensive request/response logging
⚠️ Error Handling: Consistent error responses across all endpoints
✅ Comprehensive Testing: 117+ tests with 80%+ coverage

🚀 Quick Start

Prerequisites

Python 3.11+
MongoDB 7.0+
pip

Installation

Clone the repository

git clone <repository-url>
cd mongodb-news-api

Create virtual environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install dependencies

pip install -r requirements.txt
pip install -r requirements-dev.txt  # For testing

Configure environment

cp .env.example .env
# Edit .env with your MongoDB connection and API keys
# See MONGODB_CONNECTION.md for detailed MongoDB setup

Run the application

uvicorn app.main:app --reload

The API will be available at: http://localhost:8000

📚 API Documentation

Once running, visit:

Swagger UI: http://localhost:8000/docs
ReDoc: http://localhost:8000/redoc

Response Structure

All successful API responses follow this standardized format:

{
  "success": true,
  "data": [...],
  "pagination": {
    "next_cursor": "...",
    "prev_cursor": null,
    "has_next": true,
    "has_prev": false,
    "limit": 100,
    "returned": 10
  },
  "metadata": {
    "query_time_ms": 45.32,
    "timestamp": "2025-11-22T00:15:23.445678",
    "api_version": "1.0.0"
  }
}

Error responses:

{
  "success": false,
  "error": {
    "code": "ERROR_CODE",
    "message": "Human-readable error message",
    "status": 400,
    "timestamp": "2025-11-22T00:15:23.445678"
  }
}

🔑 Authentication

All endpoints require API key authentication. Provide your API key via:

Option 1: Authorization Header

curl -H "Authorization: Bearer your-api-key" http://localhost:8000/api/v1/news

Option 2: Query Parameter

curl http://localhost:8000/api/v1/news?api_key=your-api-key

📖 API Endpoints

News Endpoints

Get News List

GET /api/v1/news

Query Parameters:

start: Filter from date (ISO 8601)
end: Filter to date (ISO 8601)
source: Filter by source (coinmarketcap, bloomberg, reuters, ...)
asset_slug: Filter by asset (bitcoin, ethereum, ...)
keyword: Search keyword
limit: Items per page (10-1000, default: 100)
cursor: Pagination cursor
sort_by: Sort field (releasedAt, title, createdAt)
order: Sort order (asc, desc)

Example:

curl "http://localhost:8000/api/v1/news?source=bloomberg&limit=50" \
  -H "Authorization: Bearer your-api-key"

Response:

{
  "success": true,
  "data": [
    {
      "slug": "bitcoin-hits-new-high",
      "title": "Bitcoin Hits New All-Time High",
      "subtitle": "BTC reaches $50,000...",
      "source": "bloomberg",
      "sourceName": "Bloomberg",
      "sourceUrl": "https://...",
      "releasedAt": "2025-02-26T12:00:00Z",
      "assets": [
        {
          "name": "Bitcoin",
          "slug": "bitcoin",
          "symbol": "BTC"
        }
      ]
    }
  ],
  "pagination": {
    "next_cursor": "eyJ...",
    "has_next": true,
    "limit": 50,
    "returned": 50
  },
  "metadata": {
    "query_time_ms": 45.32,
    "timestamp": "2025-11-22T00:15:23Z",
    "api_version": "1.0.0"
  }
}

Get Single News

GET /api/v1/news/{slug}

Example:

curl "http://localhost:8000/api/v1/news/bitcoin-hits-new-high" \
  -H "Authorization: Bearer your-api-key"

Response:

{
  "success": true,
  "data": {
    "slug": "bitcoin-hits-new-high",
    "title": "Bitcoin Hits New All-Time High",
    "subtitle": "BTC reaches $50,000...",
    "content": "<p>Full article content...</p>",
    "source": "bloomberg",
    "sourceName": "Bloomberg",
    "sourceUrl": "https://...",
    "releasedAt": "2025-02-26T12:00:00Z",
    "assets": [...],
    "createdAt": "2025-02-26T12:05:00Z",
    "updatedAt": "2025-02-26T12:05:00Z"
  },
  "metadata": {
    "query_time_ms": 12.45,
    "timestamp": "2025-11-22T00:15:23Z",
    "api_version": "1.0.0"
  }
}

Aggregation Endpoints

Get Statistics

GET /api/v1/aggregations/stats?group_by=source

Get Top Assets

GET /api/v1/aggregations/top-assets?limit=10

Get Timeline

GET /api/v1/aggregations/timeline?interval=daily

Get Source Performance

GET /api/v1/aggregations/source-performance

Health Check

GET /api/v1/health

Response:

{
  "success": true,
  "status": "healthy",
  "timestamp": "2025-11-22T00:15:23Z",
  "database": {
    "status": "connected",
    "ping_ms": 2.34
  },
  "version": "1.0.0",
  "query_time_ms": 12.45
}

🗄️ Database Schema

Collection: `news`

{
  "_id": ObjectId,
  "slug": "unique-article-slug",
  "title": "Article Title",
  "subtitle": "Article subtitle",
  "content": "<p>Full content...</p>",
  "source": "bloomberg",  // coinmarketcap, bloomberg, reuters, ...
  "sourceName": "Bloomberg",
  "sourceUrl": "https://...",
  "releasedAt": ISODate("2025-02-26T12:00:00Z"),
  "assets": [
    {
      "name": "Bitcoin",
      "slug": "bitcoin",
      "symbol": "BTC"
    }
  ],
  "createdAt": ISODate("..."),
  "updatedAt": ISODate("...")
}

Required Indexes:

// Compound index for efficient queries
db.news.createIndex({ "releasedAt": -1, "_id": -1 })

// Additional indexes
db.news.createIndex({ "slug": 1 }, { unique: true })
db.news.createIndex({ "source": 1 })
db.news.createIndex({ "assets.slug": 1 })

⚙️ Configuration

Edit .env file:

# MongoDB (with authentication)
MONGODB_URI=mongodb://localhost:27017
MONGODB_USERNAME=your_username
MONGODB_PASSWORD=your_password
MONGODB_AUTH_SOURCE=admin
MONGODB_DB_NAME=DB
MONGODB_COLLECTION_NAME=news

# Security
API_KEYS=key1,key2,key3

# Rate Limiting
RATE_LIMIT_PER_HOUR=1000

# Server
DEBUG=false
LOG_LEVEL=INFO
HOST=0.0.0.0
PORT=8000

📖 For detailed MongoDB connection options, see MONGODB_CONNECTION.md

🧪 Testing

The project includes a comprehensive test suite with 117+ tests covering:

✅ Unit tests for all endpoints
✅ Integration tests for workflows
✅ Security and authentication tests
✅ Pagination and filtering tests
✅ Error handling tests

Running Tests

Run all tests:

pytest tests/ -v

Run specific test files:

# News endpoint tests
pytest tests/test_news.py -v

# Aggregation tests
pytest tests/test_aggregations.py -v

# Integration tests
pytest tests/test_integration.py -v

# Security tests
pytest tests/test_security.py -v

Run tests by marker:

# Unit tests only
pytest tests/ -m unit -v

# Integration tests only
pytest tests/ -m integration -v

# Security tests only
pytest tests/ -m security -v

Run with coverage:

pytest tests/ --cov=app --cov-report=term-missing --cov-report=html

Expected Output:

================================ test session starts =================================
collected 117 items

tests/test_news.py::TestGetNewsList::test_get_news_without_filters PASSED      [ 1%]
tests/test_news.py::TestGetNewsList::test_get_news_with_source_filter PASSED   [ 2%]
...
tests/test_security.py::TestAuthenticationEndToEnd::test_auth_error_format PASSED [100%]

================================ 117 passed in 5.23s =================================

---------- coverage: platform linux, python 3.11.x -----------
Name                                    Stmts   Miss  Cover   Missing
---------------------------------------------------------------------
app/__init__.py                             0      0   100%
app/main.py                                45      2    96%   89-90
app/routers/news.py                        78      4    95%   165-168
app/routers/aggregations.py                95      5    95%   220-225
app/services/news_service.py              120      8    93%   145-152
app/core/pagination.py                     65      3    95%   98-100
---------------------------------------------------------------------
TOTAL                                    1245    198    84%

Test Structure

tests/
├── __init__.py
├── conftest.py                 # Test fixtures and configuration
├── test_news.py               # News endpoint tests (26 tests)
├── test_aggregations.py       # Aggregation tests (24 tests)
├── test_integration.py        # Integration workflow tests (15 tests)
├── test_security.py           # Security & auth tests (25 tests)
└── test_pagination.py         # Pagination logic tests (21 tests)

Test Coverage

News Endpoints: 26 tests covering all filtering, pagination, and error cases
Aggregations: 24 tests for stats, top assets, timeline, and performance
Integration: 15 end-to-end workflow tests
Security: 25 tests for authentication, rate limiting, and CORS
Pagination: 21 tests for cursor logic and edge cases

Overall Coverage: 84% (80%+ target achieved ✅)

🐳 Docker Support

Using Docker Compose

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

Using Dockerfile

# Build image
docker build -t mongodb-news-api .

# Run container
docker run -d -p 8000:8000 \
  -e MONGODB_URI=mongodb://host.docker.internal:27017 \
  -e API_KEYS=your-api-key \
  mongodb-news-api

📝 Development Status

Phase 1: Core & Foundation - ✅ Complete (20 files)

Phase 2: Production Ready - ✅ Complete (6 files)

Request/Response logging middleware
Rate limiting middleware (1000 req/hour)
Enhanced CORS configuration
Global error handler
Aggregation endpoints (stats, top-assets, timeline, source-performance)
Standardized response format

Phase 3: Testing & Quality - ✅ Complete (6 files)

Phase 4: GitHub Ready - 📋 Planned

CI/CD pipeline (GitHub Actions)
Docker deployment workflow
Community guidelines (CONTRIBUTING.md)
Issue & PR templates
Documentation improvements

🚀 Production Deployment

Pre-deployment Checklist

Environment variables configured
MongoDB connection tested
API keys generated and secured
Rate limiting configured appropriately
All tests passing (pytest tests/ -v)
Coverage meets requirements (80%+)
Database indexes created
Logs configured for production
Health check endpoint accessible

Deployment Options

Traditional Server (Ubuntu/CentOS)
Docker Container (Recommended)
Cloud Platforms (AWS, GCP, Azure)
Platform as a Service (Heroku, Railway, Render)

Performance Recommendations

Use MongoDB connection pooling (configured by default)
Enable API response caching for frequently accessed data
Monitor rate limiting metrics
Set up application monitoring (e.g., Sentry, DataDog)
Configure log rotation
Use reverse proxy (nginx, Caddy) for SSL/TLS

🤝 Contributing

Contributions are welcome! Please read CONTRIBUTING.md for details on:

Code style and conventions
Testing requirements (all new code must have tests)
Pull request process
Issue reporting guidelines

📄 License

This project is licensed under the MIT License - see LICENSE file for details.

🙏 Acknowledgments

Built with:

FastAPI - Modern web framework
Motor - Async MongoDB driver
MongoDB - Database
Pydantic - Data validation
pytest - Testing framework

📧 Contact

For questions or support, please open an issue on GitHub.

📚 Additional Documentation

MongoDB Connection Guide - Detailed connection setup
API Documentation - Interactive Swagger UI
Testing Guide - Comprehensive testing documentation
Changelog - Version history and updates

Version: 1.0.0
Status: Phase 3 Complete ✅
Test Coverage: 84% (117+ tests passing)
Production Ready: Yes

Name		Name	Last commit message	Last commit date
Latest commit History 21 Commits
.github		.github
app		app
docs		docs
scripts		scripts
tests		tests
.editorconfig		.editorconfig
.env.example		.env.example
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
coverage.xml		coverage.xml
docker-compose.yml		docker-compose.yml
pytest.ini		pytest.ini
requirements-dev.txt		requirements-dev.txt
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

MongoDB News API

🎯 Features

🚀 Quick Start

Prerequisites

Installation

📚 API Documentation

Response Structure

🔑 Authentication

📖 API Endpoints

News Endpoints

Get News List

Get Single News

Aggregation Endpoints

Get Statistics

Get Top Assets

Get Timeline

Get Source Performance

Health Check

🗄️ Database Schema

Collection: news

⚙️ Configuration

🧪 Testing

Running Tests

Test Structure

Test Coverage

🐳 Docker Support

Using Docker Compose

Using Dockerfile

📝 Development Status

🚀 Production Deployment

Pre-deployment Checklist

Deployment Options

Performance Recommendations

🤝 Contributing

📄 License

🙏 Acknowledgments

📧 Contact

📚 Additional Documentation

About

Resources

License

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Collection: `news`

Packages