🚀 Rubiconetic Home Lab (Kubernetes / K3s)

📖 Overview

This repository contains the GitOps Infrastructure-as-Code (IaC) manifests for my personal Kubernetes cluster ("Rubiconetic"). It serves as the production backend for my freelance automation business, hosting workflow engines, databases, and internal dashboards. The entire cluster state is reconciled automatically from this repository using FluxCD.

🏗️ Architecture

Network Strategy: The "Hybrid Access" Model

I implemented a split-horizon network security model to balance client accessibility with internal security:

• Public Access (Client-Facing): Uses Cloudflare Tunnels to expose specific webhook endpoints (n8n) to the internet without opening ports on the firewall.
• Private Access (Admin-Facing): Uses Tailscale (Mesh VPN) for all administrative tools (Pgweb, Homepage, Longhorn). These services are completely invisible to the public internet.

GitOps & Automation

• Synchronization: FluxCD watches this repository and automatically applies changes to the cluster within 1 minute.
• Secret Management: Secrets are encrypted locally using SOPS (Secrets OPerationS) and Age, allowing credentials to be safely committed to the public repository.
• Auto-Restarts: Stakater Reloader watches for secret changes and automatically restarts deployments to apply new credentials.

Infrastructure

Component	Technology	Reasoning
Orchestrator	K3s	Lightweight, production-grade Kubernetes perfect for edge hardware.
GitOps Engine	FluxCD	Automated reconciliation; "Git is the single source of truth."
Secret Encryption	SOPS + Age	Client-side encryption allowing safe storage of secrets in Git.
Ingress (Public)	Cloudflared	Zero-Trust tunneling; eliminates need for static IPs.
Ingress (Private)	Tailscale	Secure, encrypted mesh network for internal tooling access.
Storage	Longhorn	Distributed block storage for high availability.
Observability	Homepage	Centralized dashboard for service health and cluster stats.

Hosted Applications

Application	Function	Details
Forgejo	Source Code Management	Self-hosted lightweight Git service (Gitea fork).
Vaultwarden	Password Manager	Bitwarden-compatible server for credential management.
n8n	Workflow Automation	Node-based workflow automation tool.
Postgres	Database	Relational database for n8n and internal apps.
Media Stack	Torrenter + Jackett	Search & Indexing only. Actual downloads/encoding handled on external workstation via Samba/Jellyfin.
Obsidian Sync	Note Synchronization	Self-hosted LiveSync for Obsidian notes.
Monitoring	Prometheus / Grafana	Metrics collection and visualization (Kube-Prometheus-Stack).

📂 Repository Structure

This repo follows a standard Flux v2 GitOps structure:

/clusters
  └── my-cluster/# Flux Kustomization definitions & Sync config
/manifests
  ├── automation/# n8n & Postgres deployments
  ├── infrastructure/# Core networking (Cloudflared, Homepage) & Tools (Reloader)
  ├── media/         # Media stack (Torrenter, Jackett)
  ├── productivity/  # Forgejo, Obsidian Sync & other apps
  └── secrets/   # SOPS-encrypted Kubernetes Secrets

Lessons Learned & Challenges

• The GitOps Transition: Migrated from manual kubectl apply workflows to Flux. Learned to manage "Secret Zero" using SOPS and how to debug Kustomize reconciliation failures when encrypted metadata is malformed.
• DNS Hairpinning: Resolved internal service communication issues by enforcing Kubernetes internal FQDNs (svc.cluster.local) instead of relying on external Tailscale IPs for pod-to-pod talk.
• Database Migration: Successfully migrated a live production database from Coolify to K3s by managing encryption key continuity and performing a clean pg_dump/psql restore process.

🚀 Future Roadmap

• Implement GitOps (Flux + SOPS) to allow safe committing of encrypted secrets to Git.
• Configure Renovate for automated Docker image updates.
• Add historical metrics monitoring (Prometheus/Grafana).
• Cluster Hardening: Add Liveness/Readiness probes and Resource Limits to all deployments.
• Migrate to Forgejo: Migrate from GitHub to Forgejo for source code management.
• External Integrations: Add Homepage widgets for Jellyfin (External) and Home Assistant Green.
• Disaster Recovery: Configure Longhorn S3 backups.
• Database Consolidation: Migrate VaultWarden from SQLite to Postgres.
• Observability: Set up Flux Notifications (Discord/Slack).

---

Built & Maintained by Michael Rubi