GitHub Data Integration Backend

This repository contains the backend API for a robust GitHub integration dashboard. It is designed to provide organizations and developers with deep insights into their GitHub activity by securely connecting to GitHub via OAuth, synchronizing repository data, and exposing a clean, RESTful API for frontend consumption.

Features

• GitHub OAuth Integration:

  • Secure authentication with GitHub OAuth.
  • Easy login, logout, and integration removal.

• Comprehensive Data Sync:

  • Fetches and stores data from original (upstream) public repositories, including:
    • Commits
    • Pull Requests
    • Issues
    • Repository metadata
    • Organization and user information

• Real-Time Sync Status:

  • Tracks and exposes granular sync progress for each entity (commits, pulls, issues, etc.), allowing the frontend to display real-time feedback to users.

• Background Data Import:

  • Handles large-scale data imports in the background, with robust error handling, rate limit management, and the ability to abort syncs if a user removes their integration.

• Clean Data Model:

  • Uses Mongoose models for all major GitHub entities, ensuring scalable and maintainable data storage.

• Efficient Pagination & Search:

  • All endpoints support backend-driven pagination and full-text search across all relevant fields.

• Full Data Cleanup:

  • When a user removes their GitHub integration, all related data is securely and completely deleted from the database, and any running sync jobs are halted.

• Extensible & Secure:

  • Built with best practices for security, error handling, and code organization.

Tech Stack

• Node.js & Express for the REST API
• MongoDB with Mongoose for data storage and modeling
• Axios for GitHub API integration
• Express-session for secure session management
• dotenv for environment variable management

Setup Instructions

1. Clone the repository:

   git clone https://github.com/Hazique95/github-data-integration-backend.git
   cd github-data-integration-backend

2. Install dependencies:

   npm install

3. Configure environment variables:

   • Create a .env file in the root directory with the following:

     MONGODB_URI=your_mongodb_connection_string
     GITHUB_CLIENT_ID=your_github_oauth_client_id
     GITHUB_CLIENT_SECRET=your_github_oauth_client_secret
     SESSION_SECRET=your_session_secret

4. Start the server:

   npm start

Usage

• The backend exposes RESTful endpoints for authentication, data sync, and data retrieval.
• Designed to be paired with a frontend dashboard (e.g., React or Angular) for visualization.
• All data is automatically cleaned up when a user removes their integration.

License

MIT