Skip to content

server-setup.md

Latest commit

 

History

History
299 lines (245 loc) · 8.64 KB

server-setup.md

File metadata and controls

299 lines (245 loc) · 8.64 KB

Server Setup

Runbook for provisioning the Frostbyte production VM. Written against the current production box so it can be followed verbatim to rebuild from scratch.

The box

  • Host: Hetzner Cloud VM (2 vCPU, Ubuntu 24.04 LTS)
  • IP: 91.107.210.218
  • Hostname: frostbyte
  • DNS: frostbyte.tv A record → 91.107.210.218
  • SSH key: ~/.ssh/frostcrypt on the admin workstation
  • Admin user: deploy (passwordless sudo, key-only SSH)
  • App dir: /opt/frostbyte/{releases,shared}

Philosophy

Boring, reproducible, runs Elixir releases directly under systemd. No Docker, no orchestration, no config management tool. Everything in one shell script you can diff against the running box.

Bootstrap order

  1. Order a fresh Ubuntu 24.04 VM at Hetzner with your SSH public key in the root authorized_keys.
  2. Point an A record at the VM's IPv4 address.
  3. Run Phase 1 (system hardening) as root via SSH.
  4. Reconnect as deploy and run Phase 2 (Erlang + Elixir via asdf).
  5. Drop in the Caddyfile and the systemd unit once the first release exists.

Phase 1: system hardening (as root)

Runs once on first boot. Idempotent, safe to re-run.

ssh -i ~/.ssh/frostcrypt root@<new-ip> 'bash -s' <<'EOF'
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive

timedatectl set-timezone UTC
timedatectl set-ntp true

# 2 GB swap, low swappiness
if [ ! -f /swapfile ]; then
  fallocate -l 2G /swapfile
  chmod 600 /swapfile
  mkswap /swapfile
  swapon /swapfile
  echo '/swapfile none swap sw 0 0' >> /etc/fstab
  echo 'vm.swappiness=10' > /etc/sysctl.d/99-swappiness.conf
  sysctl -p /etc/sysctl.d/99-swappiness.conf
fi

apt-get update -qq
apt-get -y -qq -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade

apt-get install -y -qq \
  build-essential autoconf m4 libncurses-dev libssl-dev libreadline-dev \
  libyaml-dev libxml2-dev libxslt1-dev unzip git curl ca-certificates \
  ufw fail2ban unattended-upgrades \
  debian-keyring debian-archive-keyring apt-transport-https gnupg \
  libssh-dev libodbc2 unixodbc-dev flex bison

# deploy user
if ! id deploy &>/dev/null; then
  adduser --disabled-password --gecos "" deploy
  usermod -aG sudo deploy
fi
install -d -m 700 -o deploy -g deploy /home/deploy/.ssh
cp /root/.ssh/authorized_keys /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy/.ssh/authorized_keys
chmod 600 /home/deploy/.ssh/authorized_keys

echo 'deploy ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/deploy
chmod 440 /etc/sudoers.d/deploy
visudo -cf /etc/sudoers.d/deploy

# sshd hardening
cat > /etc/ssh/sshd_config.d/99-frostbyte.conf <<'SSHD'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AllowUsers deploy
X11Forwarding no
MaxAuthTries 3
LoginGraceTime 20
ClientAliveInterval 60
ClientAliveCountMax 3
SSHD
sshd -t
systemctl reload ssh

# firewall
ufw --force reset >/dev/null
ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
echo "y" | ufw enable

# fail2ban
cat > /etc/fail2ban/jail.local <<'F2B'
[sshd]
enabled = true
port    = 22
maxretry = 3
bantime = 3600
findtime = 600
F2B
systemctl enable --now fail2ban

# unattended-upgrades
cat > /etc/apt/apt.conf.d/20auto-upgrades <<'AUTO'
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::AutocleanInterval "7";
AUTO
systemctl enable --now unattended-upgrades

# caddy apt repo
if [ ! -f /etc/apt/sources.list.d/caddy-stable.list ]; then
  curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
    | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
  curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
    > /etc/apt/sources.list.d/caddy-stable.list
  chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
  chmod o+r /etc/apt/sources.list.d/caddy-stable.list
  apt-get update -qq
fi
apt-get install -y -qq caddy

install -d -m 755 -o deploy -g deploy /opt/frostbyte
install -d -m 755 -o deploy -g deploy /opt/frostbyte/releases
install -d -m 755 -o deploy -g deploy /opt/frostbyte/shared
install -d -m 755 -o deploy -g deploy /opt/frostbyte/shared/log

# asdf 0.18.1
ASDF_VERSION=v0.18.1
if [ ! -x /usr/local/bin/asdf ] || ! /usr/local/bin/asdf --version 2>/dev/null | grep -q "0.18.1"; then
  curl -sL "https://github.com/asdf-vm/asdf/releases/download/${ASDF_VERSION}/asdf-${ASDF_VERSION}-linux-amd64.tar.gz" -o /tmp/asdf.tar.gz
  tar -xzf /tmp/asdf.tar.gz -C /usr/local/bin/
  chmod +x /usr/local/bin/asdf
  rm /tmp/asdf.tar.gz
fi

# asdf shell integration for deploy
sudo -u deploy bash <<'DEPLOY'
cat > ~/.bashrc.asdf <<'RC'
export ASDF_DATA_DIR="$HOME/.asdf"
export PATH="$ASDF_DATA_DIR/shims:$HOME/.local/bin:$PATH"
RC
grep -qxF 'source $HOME/.bashrc.asdf' ~/.bashrc || echo 'source $HOME/.bashrc.asdf' >> ~/.bashrc
touch ~/.profile
grep -qxF 'source $HOME/.bashrc.asdf' ~/.profile || echo 'source $HOME/.bashrc.asdf' >> ~/.profile
DEPLOY

echo "phase 1 done"
EOF

After this completes, root login is disabled. All further steps run as deploy.

Phase 2: Erlang and Elixir (as deploy, in background)

Compiling Erlang from source takes roughly 10-15 minutes on a 2 vCPU box. Run it under systemd-run so it survives SSH drops and logs to the journal.

ssh -i ~/.ssh/frostcrypt deploy@<ip> 'bash -s' <<'EOF'
cat > /home/deploy/build-beam.sh <<'BUILD'
#!/bin/bash
set -euo pipefail
export HOME=/home/deploy
export ASDF_DATA_DIR="$HOME/.asdf"
export PATH="$ASDF_DATA_DIR/shims:$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin"
export KERL_CONFIGURE_OPTIONS="--without-wx --without-debugger --without-observer --without-et --without-javac --without-jinterface --enable-threads --enable-smp-support --enable-kernel-poll"
export KERL_BUILD_DOCS=no
export KERL_INSTALL_MANPAGES=no
export KERL_INSTALL_HTMLDOCS=no
export MAKEFLAGS="-j$(nproc)"

cd "$HOME"

asdf plugin add erlang https://github.com/asdf-vm/asdf-erlang.git || true
asdf plugin add elixir https://github.com/asdf-vm/asdf-elixir.git || true

ERLANG_VERSION=$(asdf latest erlang 28)
asdf install erlang "$ERLANG_VERSION"
asdf install elixir 1.19.5-otp-28
asdf set -u erlang "$ERLANG_VERSION"
asdf set -u elixir 1.19.5-otp-28
BUILD
chmod +x /home/deploy/build-beam.sh

sudo systemd-run \
  --unit=frostbyte-build \
  --uid=deploy --gid=deploy \
  --working-directory=/home/deploy \
  --setenv=HOME=/home/deploy \
  /bin/bash /home/deploy/build-beam.sh

# follow:  sudo journalctl -u frostbyte-build -f
EOF

Watch progress:

ssh deploy@<ip> sudo journalctl -u frostbyte-build -f

When you see Elixir 1.19.5 (compiled with Erlang/OTP 28) the box is ready.

Current versions (as of first bootstrap)

component version
Ubuntu 24.04 LTS
Erlang OTP 28.4.2
Elixir 1.19.5-otp-28
asdf 0.18.1
Caddy 2.11.2

Caddyfile

The production Caddyfile splits the apex (static landing) from the realtime sync subdomain so each surface has its own DNS and caching profile.

frostbyte.tv {
	encode zstd gzip
	root * /var/www/frostbyte
	file_server
}

ws.frostbyte.tv {
	reverse_proxy localhost:4000
}

DNS: both frostbyte.tv and ws.frostbyte.tv are proxied through Cloudflare (orange cloud). The CNAME for ws points at the apex.

Reload after edits: sudo systemctl reload caddy. Caddy handles Let's Encrypt certificate issuance and renewal automatically.

Landing page contents live at /var/www/frostbyte. Deployed via:

cd landing && tar -cf - . | ssh deploy@frostbyte.tv '
  set -e
  sudo rm -rf /var/www/frostbyte
  sudo mkdir -p /var/www/frostbyte
  sudo tar -C /var/www/frostbyte -xf -
  sudo chown -R root:root /var/www/frostbyte
  sudo find /var/www/frostbyte -type d -exec chmod 755 {} \;
  sudo find /var/www/frostbyte -type f -exec chmod 644 {} \;
'

systemd unit (once the first release exists)

Will live at /etc/systemd/system/frostbyte.service. Template:

[Unit]
Description=Frostbyte sync server
After=network.target

[Service]
Type=exec
User=deploy
Group=deploy
Environment=HOME=/home/deploy
Environment=LANG=en_US.UTF-8
Environment=MIX_ENV=prod
WorkingDirectory=/opt/frostbyte/current
ExecStart=/opt/frostbyte/current/bin/frostbyte start
ExecStop=/opt/frostbyte/current/bin/frostbyte stop
Restart=on-failure
RestartSec=5
KillMode=mixed
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Deploys will:

  1. Build a release locally (MIX_ENV=prod mix release)
  2. rsync to /opt/frostbyte/releases/<timestamp>/
  3. Swap the /opt/frostbyte/current symlink
  4. sudo systemctl restart frostbyte

No downtime goals for v1 (restart takes under a second, users reconnect automatically via Phoenix channel reconnection).