Airport Project

A Distributed Systems course project developed by Justine F., Bruno P., Paul V., Aymeric L.

---

Data format

{
    "idAirport": "string",
    "idSensor": "integer",
    "value": "float",
    "measure": "string",
    "date": "string"
}

---

Run

At the root, run every command in a terminal, distinct from the others.

1. Init the REDIS database:

docker-compose up

2. Run the subscriber connected to the database

go run cmd/mqtt_sub/subscriber.go configs/config_redis.json

3. Run the one writing the measures to CSV files:

go run cmd/mqtt_sub/subscriber.go configs/config_csv.json

4. Run a first publisher (the (mocked) temperature sensor installed in Nantes Airport):

go run cmd/mqtt_pub/publisher.go configs/capteur_NTE_temp.json

5. Run a second one (the (mocked) pressure sensor installed in Pointe-à-Pitre Airport):

go run cmd/mqtt_pub/publisher.go configs/capteur_PTP_pres.json

6. Launch the API:

go run pkg/api/main.go

7. For API discovery: see pkg/api/openapi.yaml file

---

Step-by-step workflow

Sensor publishers cmd/mqtt_pub/publisher.go

1. JSON config file: sensor & MQTT Broker connexion configurations
2. Connexion to the topic airport/{airportID} through the MQTT client --> MQTT broker

Data stream

1. A mocked datum is generated by trigonometric function depending on time (by calling time.Now()), with very little variation. The value is stored as a float number in the defined data structure, see the value field.
2. A SensorData is created, with {sensorID, airportID, measureType, value, timestamp} fields
3. It is then marshalled, and straight after
4. it is sent to the MQTT broker
5. The process is repeated every 10 minutes

Subscribing

1. JSON config file: sub (writing to CSV files or the REDIS DB) configuration & connexion to MQTT Broker configuration
2. FactoryControllerDAO --> SensorController REDIS or CSV
3. Listening to all topics airport/# by MQTT client--> MQTT broker

Sensor Controller DAO

Interface writing the received data (the SensorData objects) through the Write method. It's implemented according to the chosen storage method: CSV file or the REDIS DB.

Sub REDIS -> DB REDIS

The DAO embeds a REDIS client that connects to the database, using the address and the password given by the configuration file. Every call to the implemented Write method writes the SensorData (containing the value) in the DB with a unique ID (key) in the form of {sensorID}_{airportID}_{currentTime}. Eventually, it ends up by: MQTT broker ---> sub REDIS -- REDIS_client.Set (key, value) --> BDD REDIS

DB REDIS -> API

1. Connexion to the DB
2. HTTP Request to the REST API ---> route ---> handler ---> DB call
3. DB reponse ---> marshalling ---> JSON response payload

Summed up workload

• JSON config for each sensor --> PubOfAnAirportSensor -- random data --> MQTT broker <-- sub REDIS/CSV <-- JSON config for each sub
• sub REDIS --> DB --> API
• sub CSV --> CSV datalake