Godlydpatruljen

Three main components, or projects:

• API
• App
• Cloud

In addition there is a module called secrets_handler which is used for en-/decryption and up-/downloading of secrets.

Note:

• This README, and all READMEs for all components, assume that the state of this project is production.
• When using Python scripts for the different components (api.py, app.py or cloud.py) for development, end all commands with the --dev flag.
  Example: python cloud.py start-api --dev

Setup

The setup for each component is described in separate READMEs, found under the components' root. Nevertheless, there are a few steps which are considered "general", so we will go through them here.

Docker

Download and install Docker Community Edition. The API is deployed as a .jar file running inside a Docker container, and both the api and cloud components needs a running Docker daemon to create and push container images.

Google Cloud Service Account

One of the reccommended ways to gain access to the GC project and automated tools like gcloud is by using what's called a service account. This is a non-personal account where access and permissions can be set just as with any normal user account.

It is recommended to use a service account to authenticate CLI tools, even for local development.

This project depends on two service accounts: one with Owner permissions to use during development, and one with restricted permissions for automated use.

Something, something permissions

The (restricted) service account needs a certain set of permissions to work properly:

• Project permission

  • cloudkms.keyRings.list

• Bucket permissions
  (The artifact bucket / registry where Docker images are pushed to.)

  • Manually set the service account as Storage Admin for the bucket.

• Crypto Key permissions

  • cloudkms.cryptoKeyVersions.get
  • cloudkms.cryptoKeyVersions.list
  • cloudkms.cryptoKeyVersions.useToDecrypt
  • cloudkms.cryptoKeys.get
  • cloudkms.cryptoKeys.list
  • cloudkms.keyRings.get
  • cloudkms.keyRings.list
  • resourcemanager.projects.get

Note:
Individual permissions can't be set in GCloud, but must be set indirectly via a role. It is recommended to create a new role to list key rings, and another for the usage of crypto keys.

This link provides some help with setting and removing roles via the CLI.

Creating service accounts

1. Log in to Google Cloud Platform.
2. Via the top-left triple-dash menu, open IAM & admin > Service accounts.
3. Click Create Service Account in the top bar.
   1. Create a name and an ID for the account.
      Note: The ID is what will be used for identifying the account, while "Name" is just a convenience tag.
   2. Set a role for the account. This role will have "Project" as scope.
      Use "Owner" for the development account.
      Use the custom role with the cloudkms.keyRings.list permission for the automated account.
4. Set permissions for the automated service account according to the steps above.

Authenticate with a service account

1. Log in to Google Cloud Platform.
2. Via the top-left triple-dash menu, open IAM & admin > Service accounts.
3. Find the service account with "Owner" permissions, click on the triple-dot menu on its far right, and click on Create key.
4. Select JSON and click Create.
5. Download the JSON file to a desired location on your local hard drive.
6. Create a permanent environment variable called GOOGLE_APPLICATION_CREDENTIALS and set its value to be the full path of your JSON service account key.
7. Now create and download a key for the automated service account, and save the file as hlf/secrets/credentials.json.

Python

Python is used for a good many scripts and commands in this project, and is required for everything to work properly.

1. Download and install Python. The scripts were written and tested with Python 3.6.6, so that's the recommended version.
2. Upgrade pip to the latest version:
   python -m pip install --upgrade pip
3. Make sure your current directory is hlf, and create a new virtual environment for Python:
   python -m venv venv
   (the first venv is the command, and the second is the folder (environment) name).
   • venv is the recommended tool for creating virtual environments after Python 3.5. virtualenv is deprecated after Python 3.6.
   • For more information about venv, and how to (de)activate the environment for your system, visit the python venv user guide
4. Activate your new environment
5. Install the required packages:
   pip install -r requirements.txt
   • If, for some reason, pip isn't installing the packages in the correct location:
     1. Check (and triple-check) that your environment is activated
     2. Install the requirements with the prefix python -m:
        python -m pip install -r requirements.txt

Secrets

Secrets are encrypted and stored in a bucket on Google Cloud Platform. The uploading and downloading of secrets are done in a similar matter: by using either the upload_secret.py or download_secret.py script found in the secrets_handler folder. (Usage is documented in the Secrets README.)

The most important file to download is the one called secrets.json.

Deployment

The pipeline is as follows:

1. Follow the steps in api to
   1. Build the server application as a .jar
   2. Build a Docker image
2. Follow the steps in cloud to
   1. Push the Docker image
   2. Build a server image
   3. Setup the infrastructure with Terraform, and start a server instance
   4. Start the server application
3. Follow the steps in app to build the mobile application to Android and/or iOS