The Event Scraper service is a part of the Events APIs team. The service is responsible for scraping data and uploading them to an S3 bucket from the following CMU data sources:
-
25Live: the room reservation data for the next 7 days.
-
Handshake: the Handshake events for the next 30 days.
-
Tartan Connect: the Tartan Connect events.
Then the Event Scraper service notifies the Railway services defined in the
NOTIF_CONFIG environment variable to restart their deployments,
which are responsible for pulling the data from S3 bucket and processing them.
You can regenerate this diagram by pasting the linked code
into Mermaid.
See Contributing Guide for instructions on how to access the S3 bucket credentials.
See internal documentation on how this service is deployed.
Open the project with Dev Container. Populate the environment variables following the env.ts file. Most secrets are stored in Railway but you would need to supply your own CMU credentials. To run the service locally, run the following command:
bun src/index.ts