OpenDataMask/docs/website/deployment.html at main · MaximumTrainer/OpenDataMask · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Deployment – OpenDataMask</title>
  <link rel="stylesheet" href="style.css" />
</head>
<body>
  <nav class="nav">
    <div class="nav-inner">
      <a href="index.html" class="nav-brand">🛡️ OpenDataMask</a>
      <ul class="nav-links">
        <li><a href="index.html">Home</a></li>
        <li><a href="guide.html">User Guide</a></li>
        <li><a href="deployment.html" class="active">Deployment</a></li>
        <li><a href="api.html">API Reference</a></li>
        <li><a href="https://github.com/MaximumTrainer/OpenDataMask" target="_blank">GitHub ↗</a></li>
      </ul>
    </div>
  </nav>

  <section class="page-hero">
    <h1>Deployment &amp; Installation</h1>
    <p>Docker Compose, Terraform on AWS, Kubernetes — deploy anywhere.</p>
  </section>

  <div class="content-layout">
    <aside class="sidebar">
      <nav class="sidebar-nav">
        <h4>Contents</h4>
        <ul>
          <li><a href="#docker-compose">Docker Compose</a></li>
          <li><a href="#docker-images">Docker Images</a></li>
          <li><a href="#environment">Environment Variables</a></li>
          <li><a href="#database">Database Setup</a></li>
          <li><a href="#terraform">Terraform (AWS)</a></li>
          <li><a href="#kubernetes">Kubernetes</a></li>
          <li><a href="#cicd">CI / CD Pipelines</a></li>
          <li><a href="#security">Security Notes</a></li>
        </ul>
      </nav>
    </aside>

    <main class="main-content">
      <h2 id="docker-compose">Docker Compose (Recommended)</h2>
      <p>The simplest way to run OpenDataMask in any environment:</p>
      <pre><code>git clone https://github.com/MaximumTrainer/OpenDataMask.git
cd OpenDataMask

# Generate and export secrets
export JWT_SECRET=$(openssl rand -base64 32)
export ENCRYPTION_KEY=$(openssl rand -base64 32 | head -c 32)

# Start all services
docker-compose up -d

# Check status
docker-compose ps
docker-compose logs -f backend</code></pre>
      <p>Services started:</p>
      <ul>
        <li><strong>frontend</strong> — Vue 3 UI served by nginx on port <code>80</code></li>
        <li><strong>backend</strong> — Spring Boot REST API on port <code>8080</code></li>
        <li><strong>postgres</strong> — PostgreSQL 16 database on port <code>5432</code></li>
      </ul>

      <h2 id="docker-images">Docker Images</h2>
      <p>Pre-built images are published to the GitHub Container Registry on every push to <code>main</code>:</p>
      <pre><code># Backend
docker pull ghcr.io/maximumtrainer/opendatamask/backend:latest

# Frontend
docker pull ghcr.io/maximumtrainer/opendatamask/frontend:latest

# CLI
docker pull ghcr.io/maximumtrainer/opendatamask/cli:latest</code></pre>

      <h3>Building locally</h3>
      <pre><code>docker build -t opendatamask-backend ./backend
docker build -t opendatamask-frontend ./frontend
docker build -t opendatamask-cli ./cli</code></pre>

      <h2 id="environment">Environment Variables</h2>
      <table>
        <tr><th>Variable</th><th>Required</th><th>Default</th><th>Description</th></tr>
        <tr><td><code>DATABASE_URL</code></td><td>Yes</td><td>—</td><td>JDBC URL for PostgreSQL metadata store</td></tr>
        <tr><td><code>DATABASE_USERNAME</code></td><td>Yes</td><td>—</td><td>PostgreSQL username</td></tr>
        <tr><td><code>DATABASE_PASSWORD</code></td><td>Yes</td><td>—</td><td>PostgreSQL password</td></tr>
        <tr><td><code>JWT_SECRET</code></td><td>Yes</td><td>—</td><td>JWT signing secret. Generate with <code>openssl rand -base64 32</code></td></tr>
        <tr><td><code>ENCRYPTION_KEY</code></td><td>Yes</td><td>—</td><td>Credential encryption key, exactly 16 or 32 characters</td></tr>
        <tr><td><code>SERVER_PORT</code></td><td>No</td><td>8080</td><td>Backend HTTP listen port</td></tr>
        <tr><td><code>JWT_EXPIRATION</code></td><td>No</td><td>86400000</td><td>Token expiry in milliseconds (default 24 h)</td></tr>
        <tr><td><code>MONGODB_URI</code></td><td>No</td><td>—</td><td>MongoDB connection URI (only needed when masking MongoDB sources)</td></tr>
      </table>

      <h2 id="database">Database Setup</h2>
      <p>OpenDataMask uses <strong>PostgreSQL 15+</strong> for its metadata store. On first startup, Hibernate automatically creates the required schema (<code>ddl-auto: update</code>). No manual migration is needed.</p>
      <pre><code># Create database manually (if not using docker-compose)
psql -U postgres -c "CREATE DATABASE opendatamask;"
psql -U postgres -c "CREATE USER opendatamask WITH PASSWORD 'secret';"
psql -U postgres -c "GRANT ALL PRIVILEGES ON DATABASE opendatamask TO opendatamask;"</code></pre>

      <h2 id="terraform">Terraform (AWS)</h2>
      <p>The <code>infra/</code> directory provides Terraform configuration to provision a complete AWS environment — VPC, EC2 instance, security groups, Elastic IP, and S3/DynamoDB remote state. Everything runs as docker-compose on a single <strong>t3.small</strong> EC2 instance (Amazon Linux 2023), keeping costs low while remaining production-upgradeable.</p>

      <h3>Prerequisites</h3>
      <ul>
        <li>Terraform 1.6+ installed — <a href="https://developer.hashicorp.com/terraform/install" target="_blank">install guide</a></li>
        <li>AWS account with IAM credentials (EC2, VPC, S3, DynamoDB permissions)</li>
        <li>AWS CLI configured: <code>aws configure</code></li>
      </ul>

      <h3>One-time: Bootstrap Remote State</h3>
      <pre><code># Create S3 bucket for Terraform state
aws s3api create-bucket --bucket my-opendatamask-tfstate --region us-east-1
aws s3api put-bucket-versioning \
  --bucket my-opendatamask-tfstate \
  --versioning-configuration Status=Enabled

# Create DynamoDB table for state locking
aws dynamodb create-table \
  --table-name opendatamask-tf-locks \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST</code></pre>

      <h3>Deploy</h3>
      <pre><code>cd infra
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars — add your SSH public key

terraform init \
  -backend-config="bucket=my-opendatamask-tfstate" \
  -backend-config="dynamodb_table=opendatamask-tf-locks" \
  -backend-config="region=us-east-1"

terraform plan
terraform apply

# Get the server's public IP
terraform output server_public_ip</code></pre>

      <h3>GitHub Secrets for CI/CD Pipeline</h3>
      <p>Configure in <strong>GitHub → Settings → Secrets and variables → Actions</strong>:</p>
      <table>
        <tr><th>Secret</th><th>Description</th></tr>
        <tr><td><code>AWS_ACCESS_KEY_ID</code></td><td>AWS IAM access key</td></tr>
        <tr><td><code>AWS_SECRET_ACCESS_KEY</code></td><td>AWS IAM secret key</td></tr>
        <tr><td><code>AWS_REGION</code></td><td>AWS region (e.g. <code>us-east-1</code>)</td></tr>
        <tr><td><code>EC2_SSH_PRIVATE_KEY</code></td><td>PEM private key for SSH deploys</td></tr>
        <tr><td><code>EC2_SSH_PUBLIC_KEY</code></td><td>Matching SSH public key (stored in EC2)</td></tr>
        <tr><td><code>JWT_SECRET</code></td><td>32+ char JWT signing secret</td></tr>
        <tr><td><code>ENCRYPTION_KEY</code></td><td>32 char field encryption key</td></tr>
        <tr><td><code>TF_STATE_BUCKET</code></td><td>S3 bucket for Terraform state</td></tr>
        <tr><td><code>TF_STATE_DYNAMODB_TABLE</code></td><td>DynamoDB table for state locking</td></tr>
      </table>

      <h2 id="kubernetes">Kubernetes</h2>
      <p>A basic Kubernetes deployment uses standard <code>Deployment</code> and <code>Service</code> resources. Store secrets with <code>kubectl create secret</code>:</p>
      <pre><code>kubectl create secret generic opendatamask-secrets \
  --from-literal=JWT_SECRET="$(openssl rand -base64 32)" \
  --from-literal=ENCRYPTION_KEY="$(openssl rand -base64 32 | head -c 32)" \
  --from-literal=DATABASE_PASSWORD="your-db-password"</code></pre>
      <p>Reference the secret in your Deployment <code>envFrom</code> block. A PostgreSQL StatefulSet or a managed cloud database (AWS RDS, Azure Database for PostgreSQL, Google Cloud SQL) is recommended for production.</p>

      <h2 id="cicd">CI / CD Pipelines</h2>
      <p>OpenDataMask ships with five GitHub Actions workflows delivering a complete terraform → docker → deploy → verify pipeline:</p>
      <table>
        <tr><th>Workflow</th><th>File</th><th>Trigger</th><th>Purpose</th></tr>
        <tr><td>CI</td><td><code>ci.yml</code></td><td>push / PR to main</td><td>Build, lint, test backend + frontend + CLI</td></tr>
        <tr><td>Docker Build</td><td><code>docker.yml</code></td><td>push to main</td><td>Build and push images to GHCR</td></tr>
        <tr><td><strong>Deploy</strong></td><td><code>deploy.yml</code></td><td>after Docker Build</td><td><strong>Full pipeline: terraform → docker → SSH deploy → health verify</strong></td></tr>
        <tr><td>Deployment Verification</td><td><code>verify-deployment.yml</code></td><td>manual / after CI</td><td>Spring Boot smoke tests + optional live server health check</td></tr>
        <tr><td>CodeQL</td><td><code>codeql.yml</code></td><td>push / PR / weekly</td><td>Static security analysis for Kotlin, JS/TS, Go</td></tr>
      </table>

      <h3>Deploy Pipeline Flow</h3>
      <pre><code>push to main
  └─► CI (tests pass)
        └─► Docker Build &amp; Push (images → GHCR)
              └─► deploy.yml:
                    ├─ Job 1: terraform apply   ← provision/update AWS infra
                    ├─ Job 2: docker push        ← parallel image build
                    ├─ Job 3: SSH deploy         ← docker-compose pull &amp;&amp; up
                    └─ Job 4: verify             ← curl /actuator/health → 200 ✅</code></pre>

      <p>GitHub <strong>Environments</strong> (<code>staging</code>, <code>production</code>) track each deployment — enabling Copilot deployment status, history, and environment URLs in the GitHub UI.</p>
      <p>Run smoke tests manually from the <em>Actions</em> tab → <em>Deployment Verification</em> → <em>Run workflow</em>.</p>

      <h2 id="security">Security Notes</h2>
      <ul>
        <li>Never commit <code>JWT_SECRET</code> or <code>ENCRYPTION_KEY</code> to source control. Use GitHub Secrets or a secrets manager.</li>
        <li>The health endpoint (<code>/actuator/health</code>) is publicly accessible by default. Restrict it at the network level in production.</li>
        <li>All connection passwords are encrypted at rest using AES-256 keyed by <code>ENCRYPTION_KEY</code>.</li>
        <li>HTTPS termination should be handled by a reverse proxy (nginx, Cloudflare, ALB) in front of the backend.</li>
        <li>JWT tokens expire after 24 hours by default (<code>JWT_EXPIRATION</code>).</li>
      </ul>
    </main>
  </div>

  <footer class="footer">
    <div class="container">
      <p>🛡️ <strong>OpenDataMask</strong> — Open-source data masking platform.</p>
      <p><a href="https://github.com/MaximumTrainer/OpenDataMask" target="_blank">GitHub</a> · <a href="guide.html">User Guide</a> · <a href="deployment.html">Deployment</a> · <a href="api.html">API</a></p>
    </div>
  </footer>
</body>
</html>