diff --git a/.gitignore b/.gitignore index 5bdd3e7..01a9108 100644 --- a/.gitignore +++ b/.gitignore @@ -3,4 +3,5 @@ rssagg .env thunder-tests/ vendor/* -modules.txt \ No newline at end of file +modules.txt +*.drawio \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..72431f5 --- /dev/null +++ b/README.md @@ -0,0 +1,135 @@ +## RSS Aggregator API + +- This project develops an API that allows users to authenticate, scrape RSS feeds, follow feeds of their choice, and view posts from the feeds they follow +- The API is tested, dockerised and available on [Docker Hub](https://hub.docker.com/repository/docker/timee98642/rss-agg-api/general) +- Then the API is used in a local Kubernetes application, ending with dashboards for monitoring Kubernetes and the API + +*Tech: Go, PostgreSQL, GitHub Actions, Docker, Kubernetes, Prometheus, Grafana* + +### Table of Contents: + +- [API](#api) +- [CI/CD](#cicd) +- [Kubernetes](#kubernetes) +- [Monitoring & Metrics](#monitoring--metrics) +- [Setup](#setup) +- [Future](#future) + +## API + +![api-docs](project-info/api.svg) + +- There are 4 main Entities (User, Feed, Post, FeedFollows) alongside their schemas. Each is in its own table in PostgreSQL + +**User queries on `/users`** +- GET (auth): returns a User entity +- POST: creates a User and returns a User + +**Feed queries on `/feeds`** +- GET: returns list of Feed entities +- POST (auth): takes in a name and url to scrape its RSS feed and returns a Feed + +**Post queries on `/posts`** +- GET (auth): (optionally takes pagination params) returns a list of Post entities for the authenticated user + +**FeedFollows queries on `/feed_follows`** +- GET (auth): returns followed feeds for the authenticated user +- POST (auth): the authenticated User follows a Feed and returns a FeedFollow +- DELETE (auth): the authenticated User unfollows a feed + +**There are 2 middlewares** +- Authentication (on selected actions) +- Logging request/reponses to the database for monitoring (on all) + +(I also tried creating a docs with swagger. The result looks similar to the auto-generated docs by FastAPI - picture in the docs directory) + +## CI/CD + +![cicd-flow](project-info/cicd-flow.svg) + +The dockerised API is lint, tested, and pushed to [Docker Hub](https://hub.docker.com/repository/docker/timee98642/rss-agg-api/general) using GitHub Actions. + +## Kubernetes + +![k8s-flow](project-info/k8s-flow.svg) + +**API Traffic Flow** +- Users interact with the `api-service` (LoadBalancer), which forwards traffic to the `api pod` +- The api pods handle the request and may need to query the database (`db`) + +**Database Access with NetworkPolicy** +- The `api pod` connects to the `db-service`, which routes traffic to `db pods` +- The `db-access-policy` NetworkPolicy allows the `api pod` to access the `db` + +**Storage for Database** +- The `db pod` uses Persistent Volume Claims (PVCs) to request storage +- The storage is backed by a Persistent Volume (PV) + +**Secrets** +- Database connection information is passed to both `api pod` and `db pod` + +## Monitoring & Metrics + +- I setup Prometheus (with Helm) for K8s cluster monitoring and metrics. The information is sent to Grafana (setup with Helm) to create dashboards. Here are dashboards pics from the `api pod` and `db pod` + +`api pod` view + +![api-pod-dashboard](project-info/api-pod-dashboard.png) + +`db pod` view + +![db-pod-dashboard](project-info/db-pod-dashboard.png) + +- I also setup a dashboard to monitor the API traffic + +![api-logs-dashboard](project-info/api-logs-dashboard.png) + + +#### Setup Kubernetes + +```bash +kubectl apply -f kubernetes/. + +# check everything is good +kubectl get all + +# expose API to local +kubectl port-forward api-pod-name 8080 +``` + +#### Setup Monitoring + +```bash +helm repo add prometheus-community https://prometheus-community.github.io/helm-charts + +helm repo update + +helm install my-prometheus prometheus-community/prometheus + +helm repo add grafana https://grafana.github.io/helm-charts + +helm repo update + +helm install my-grafana grafana-grafana + +# check pods/svc with kubectl are running... + +# get grafana pwd and save it for login +kubectl get secret -n default my-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo + +# port-forward grafana to localhost +kubectl port-forward service/my-grafana 3000:80 --address='0.0.0.0' + +# visit 0.0.0.0:3000 and use admin/pwd +``` + +You should see something like if everything is ok: + +![k8s-all](project-info/k8s-all.png) + +### Future + +- send API logs to a dedicated database and/or create a main/copy database pod for writes only to the main, and the copy reads them and is used for dashboards +- try using an ingress controller for exposing the API with K8s +- learn more about Go and implement things like the repository pattern for the API +- add caching to the API diff --git a/docs/api-docs.png b/docs/api-docs.png new file mode 100644 index 0000000..8368f58 Binary files /dev/null and b/docs/api-docs.png differ diff --git a/entrypoint.sh b/entrypoint.sh index b67ef56..8a3b43b 100644 --- a/entrypoint.sh +++ b/entrypoint.sh @@ -1,7 +1,7 @@ #!/bin/bash set -e -echo "Waiting for the database to start... (sleep 30)" +echo "Waiting for the database to start... (sleeping 30)" sleep 30 echo "Running database migrations..." diff --git a/grafana/api-logs-dashboard.json b/grafana/api-logs-dashboard.json new file mode 100644 index 0000000..e34b6ea --- /dev/null +++ b/grafana/api-logs-dashboard.json @@ -0,0 +1,984 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 6, + "links": [], + "panels": [ + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [ + { + "options": { + "failure_count": { + "color": "red", + "index": 0 + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "failure_count" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 6, + "w": 10, + "x": 0, + "y": 0 + }, + "id": 7, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n date_trunc('minute', timestamp) AS time_bucket, \n COUNT(CASE WHEN status BETWEEN '200' AND '299' THEN 1 END) AS success_count,\n COUNT(CASE WHEN status >= '400' THEN 1 END) AS failure_count\nFROM logs\nGROUP BY time_bucket\nORDER BY time_bucket;\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "logs" + } + ], + "title": "Success vs Failed Requests Over Time", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisGridShow": true, + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "smooth", + "lineStyle": { + "fill": "solid" + }, + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [ + { + "options": { + "100": { + "color": "dark-red", + "index": 0 + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "#EAB839", + "value": 100 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 10, + "x": 10, + "y": 0 + }, + "id": 1, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n date_trunc('minute', timestamp) AS time_bucket, \n COUNT(*) AS request_count\nFROM logs\nGROUP BY time_bucket\nORDER BY time_bucket;\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "logs" + } + ], + "title": "Request Count Over Time", + "type": "timeseries" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "text", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 4, + "x": 20, + "y": 0 + }, + "id": 3, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n avg(duration_ms)\nFROM logs\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Avg Request Duration (ms)", + "type": "stat" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "inspect": false + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 5, + "w": 12, + "x": 0, + "y": 6 + }, + "id": 5, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n url, \n AVG(duration_ms) AS avg_duration_ms,\n COUNT(*) AS request_count,\n PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY duration_ms) AS pct_95_duration_ms\nFROM logs\nWHERE url NOT LIKE '/swagger%' AND url NOT LIKE '/favicon%'\nGROUP BY url\nORDER BY avg_duration_ms DESC;\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Slowest Endpoints", + "type": "table" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "#C4162A", + "mode": "palette-classic-by-name" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 80, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineWidth": 1, + "scaleDistribution": { + "type": "linear" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [ + { + "options": { + "from": 400, + "result": { + "color": "red", + "index": 0 + }, + "to": 1000 + }, + "type": "range" + }, + { + "options": { + "from": 200, + "result": { + "color": "green", + "index": 1 + }, + "to": 299 + }, + "type": "range" + }, + { + "options": { + "from": 100, + "result": { + "color": "super-light-green", + "index": 2 + }, + "to": 199 + }, + "type": "range" + }, + { + "options": { + "from": 300, + "result": { + "color": "super-light-green", + "index": 3 + }, + "to": 399 + }, + "type": "range" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 11, + "w": 6, + "x": 12, + "y": 6 + }, + "id": 4, + "options": { + "barRadius": 0, + "barWidth": 0.97, + "colorByField": "status", + "fullHighlight": false, + "groupWidth": 0.7, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "orientation": "horizontal", + "showValue": "auto", + "stacking": "none", + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + }, + "xTickLabelRotation": 0, + "xTickLabelSpacing": 0 + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n status, \n COUNT(*) * 100.0 / SUM(COUNT(*)) OVER () AS count\nFROM logs\nGROUP BY status\nORDER BY count DESC;\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Response Status %", + "type": "barchart" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + } + }, + "fieldMinMax": false, + "mappings": [ + { + "options": { + "GET": { + "color": "blue", + "index": 0 + } + }, + "type": "value" + } + ] + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "GET" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "POST" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "purple", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 6, + "x": 18, + "y": 6 + }, + "id": 2, + "options": { + "displayLabels": [ + "value", + "name" + ], + "legend": { + "displayMode": "list", + "placement": "right", + "showLegend": true + }, + "pieType": "pie", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n method AS \"label\", \n COUNT(method) * 100.0 / SUM(COUNT(method)) OVER () AS \"percentage\"\nFROM logs\nGROUP BY method\nORDER BY \"percentage\" DESC;\n", + "refId": "A", + "sql": { + "columns": [ + { + "name": "COUNT", + "parameters": [ + { + "name": "method", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "method", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "name": "method", + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "logs" + } + ], + "title": "HTTP Methods %", + "type": "piechart" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "red", + "mode": "fixed" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 79, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineWidth": 2, + "scaleDistribution": { + "type": "linear" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "fieldMinMax": false, + "mappings": [ + { + "options": { + "0-9ms": { + "color": "green", + "index": 0 + }, + "10-19ms": { + "color": "semi-dark-green", + "index": 1 + }, + "20-29ms": { + "color": "orange", + "index": 2 + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "request_count" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "transparent", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 6, + "w": 12, + "x": 0, + "y": 11 + }, + "id": 8, + "options": { + "barRadius": 0.05, + "barWidth": 0.74, + "colorByField": "bucket_label", + "fullHighlight": false, + "groupWidth": 0.7, + "legend": { + "calcs": [], + "displayMode": "table", + "placement": "right", + "showLegend": false + }, + "orientation": "horizontal", + "showValue": "auto", + "stacking": "none", + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + }, + "xField": "bucket_label", + "xTickLabelRotation": 0, + "xTickLabelSpacing": 0 + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "WITH cte AS (\n SELECT \n FLOOR(duration_ms / 10) * 10 AS duration_bucket, \n COUNT(*) AS request_count,\n CASE\n WHEN FLOOR(duration_ms / 10) * 10 = 0 THEN '0-9ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 10 THEN '10-19ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 20 THEN '20-29ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 30 THEN '30-39ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 40 THEN '40-49ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 50 THEN '50-59ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 60 THEN '60-69ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 70 THEN '70-79ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 80 THEN '80-89ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 90 THEN '90-99ms'\n WHEN FLOOR(duration_ms / 10) * 10 = 100 THEN '100ms'\n ELSE 'Other' \n END AS bucket_label\n FROM logs\n WHERE duration_ms <= 100\n GROUP BY duration_bucket\n)\nSELECT \n bucket_label, \n -- request_count, \n request_count * 100.0 / SUM(request_count) OVER () AS request_count\nFROM cte\nORDER BY duration_bucket;\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Response (ms) Distribution %", + "type": "barchart" + }, + { + "datasource": { + "type": "grafana-postgresql-datasource", + "uid": "befp4e3pxuscge" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "inspect": false + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 6, + "x": 18, + "y": 13 + }, + "id": 6, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT \n caller_user_id, \n COUNT(*) AS request_count\nFROM logs\nGROUP BY caller_user_id\nORDER BY request_count DESC\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Top Users", + "type": "table" + } + ], + "preload": false, + "refresh": "5s", + "schemaVersion": 40, + "tags": [], + "templating": { + "list": [] + }, + "time": { + "from": "now-6h", + "to": "now" + }, + "timepicker": {}, + "timezone": "browser", + "title": "API Logs", + "uid": "aefpb9ex43n5sc", + "version": 11, + "weekStart": "" +} \ No newline at end of file diff --git a/grafana/k8s-pods-dashboard.json b/grafana/k8s-pods-dashboard.json new file mode 100644 index 0000000..1b0e38f --- /dev/null +++ b/grafana/k8s-pods-dashboard.json @@ -0,0 +1,1255 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "description": "Dashboard for kubernetes pod", + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 5, + "links": [], + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 168, + "panels": [], + "title": "Pod $pod", + "type": "row" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "decimals": 2, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "blue", + "value": null + } + ] + }, + "unit": "s" + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 9, + "x": 0, + "y": 1 + }, + "id": 179, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "mean" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "time()-kube_pod_start_time{namespace=\"$namespace\", pod=\"$pod\", app=\"\"}", + "instant": true, + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "title": "Uptime", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "mappings": [ + { + "options": { + "0": { + "text": "Good" + }, + "1": { + "text": "Bad" + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 1 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 8, + "x": 9, + "y": 1 + }, + "id": 173, + "options": { + "colorMode": "background", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(kube_pod_container_status_waiting{namespace=\"$namespace\", pod=\"$pod\", reason!~\"ContainerCreating\"})", + "instant": true, + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "title": "Health", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "yellow", + "value": 2 + }, + { + "color": "red", + "value": 5 + } + ] + }, + "unit": "none" + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 7, + "x": 17, + "y": 1 + }, + "id": 175, + "options": { + "colorMode": "background", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(round(increase(kube_pod_container_status_restarts_total{pod=\"$pod\"}[$duration])))", + "format": "time_series", + "instant": true, + "interval": "", + "intervalFactor": 1, + "legendFormat": "", + "refId": "A" + } + ], + "title": "Restart", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "decimals": 2, + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "percentunit" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "CPU(%)" + }, + "properties": [ + { + "id": "unit", + "value": "percentunit" + }, + { + "id": "thresholds", + "value": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 0.8 + } + ] + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Memory(%)" + }, + "properties": [ + { + "id": "unit", + "value": "percentunit" + }, + { + "id": "thresholds", + "value": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 0.8 + } + ] + } + } + ] + } + ] + }, + "gridPos": { + "h": 6, + "w": 12, + "x": 0, + "y": 5 + }, + "id": 171, + "options": { + "displayMode": "lcd", + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "maxVizHeight": 300, + "minVizHeight": 10, + "minVizWidth": 0, + "namePlacement": "auto", + "orientation": "horizontal", + "reduceOptions": { + "calcs": [ + "last" + ], + "fields": "", + "values": false + }, + "showUnfilled": true, + "sizing": "auto", + "valueMode": "color" + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(rate(container_cpu_usage_seconds_total{namespace=\"$namespace\", pod=\"$pod\"}[$duration])) / sum(kube_pod_container_resource_requests{namespace=\"$namespace\", pod=\"$pod\", resource=\"cpu\"})", + "interval": "", + "legendFormat": "CPU(%)", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(container_memory_usage_bytes{namespace=\"$namespace\", pod=\"$pod\"}) / sum(kube_pod_container_resource_requests{namespace=\"$namespace\", pod=\"$pod\", resource=\"memory\"})", + "interval": "", + "legendFormat": "Memory(%)", + "range": true, + "refId": "B" + } + ], + "title": "Resource Usage", + "type": "bargauge" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "noValue": "∞", + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "rgb(255, 255, 255)", + "value": null + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Memory" + }, + "properties": [ + { + "id": "unit", + "value": "decbytes" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "CPU core" + }, + "properties": [ + { + "id": "unit", + "value": "Core" + } + ] + } + ] + }, + "gridPos": { + "h": 6, + "w": 6, + "x": 12, + "y": 5 + }, + "id": 177, + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "horizontal", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "mean" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(kube_pod_container_resource_requests{namespace=\"$namespace\", pod=\"$pod\", resource=\"cpu\"})", + "interval": "", + "legendFormat": "CPU core", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(kube_pod_container_resource_requests{namespace=\"$namespace\", pod=\"$pod\", resource=\"memory\"})", + "interval": "", + "legendFormat": "Memory", + "range": true, + "refId": "B" + } + ], + "title": "Resource Request", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "mappings": [], + "noValue": "∞", + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "rgb(255, 255, 255)", + "value": null + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Memory" + }, + "properties": [ + { + "id": "unit", + "value": "decbytes" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "CPU core" + }, + "properties": [ + { + "id": "unit", + "value": "Core" + } + ] + } + ] + }, + "gridPos": { + "h": 6, + "w": 6, + "x": 18, + "y": 5 + }, + "id": 183, + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "horizontal", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "mean" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(kube_pod_container_resource_limits{namespace=\"$namespace\", pod=\"$pod\", resource=\"cpu\"})", + "interval": "", + "legendFormat": "CPU core", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(kube_pod_container_resource_limits{namespace=\"$namespace\", pod=\"$pod\", resource=\"memory\"})", + "interval": "", + "legendFormat": "Memory", + "range": true, + "refId": "B" + } + ], + "title": "Resource Limit", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "none" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "in" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "#73BF69", + "mode": "fixed" + } + }, + { + "id": "custom.transform", + "value": "negative-Y" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "out" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "#5794F2", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 11 + }, + "id": 180, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(container_memory_swap{namespace=\"$namespace\", pod=\"$pod\"}) by (pod)", + "interval": "", + "intervalFactor": 1, + "legendFormat": "Swap", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(rate(container_memory_failcnt{namespace=\"$namespace\", pod=\"$pod\"}[$duration])) by (pod)", + "interval": "", + "legendFormat": "Hits limits", + "range": true, + "refId": "B" + } + ], + "title": "Memory (Swap/Hits Limits)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "#EAB839", + "value": 1 + }, + { + "color": "red", + "value": 2 + } + ] + }, + "unit": "none" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 11 + }, + "id": 184, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(round(increase(container_oom_events_total{namespace=\"$namespace\", pod=\"$pod\"}[$duration])))", + "instant": false, + "legendFormat": "OOM", + "range": true, + "refId": "A" + } + ], + "title": "Out of Memory", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisGridShow": true, + "axisLabel": "In | Out", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "bars", + "fillOpacity": 100, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "decbytes" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "In" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "#73BF69", + "mode": "fixed" + } + }, + { + "id": "custom.transform", + "value": "negative-Y" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Out" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "#5794F2", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 19 + }, + "id": 169, + "options": { + "legend": { + "calcs": [ + "min", + "mean", + "max" + ], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(rate(container_network_receive_bytes_total{namespace=\"$namespace\", pod=\"$pod\"}[$duration]))", + "interval": "", + "intervalFactor": 1, + "legendFormat": "In", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(rate(container_network_transmit_bytes_total{namespace=\"$namespace\", pod=\"$pod\"}[$duration]))", + "interval": "", + "legendFormat": "Out", + "range": true, + "refId": "B" + } + ], + "title": "Network I/O", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "points", + "fillOpacity": 40, + "gradientMode": "opacity", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Thread" + }, + "properties": [ + { + "id": "custom.drawStyle", + "value": "bars" + }, + { + "id": "custom.fillOpacity", + "value": 100 + }, + { + "id": "color", + "value": { + "fixedColor": "orange", + "mode": "fixed" + } + }, + { + "id": "custom.lineWidth", + "value": 0 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 19 + }, + "id": 182, + "options": { + "legend": { + "calcs": [ + "min", + "mean", + "max" + ], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "11.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "editorMode": "code", + "expr": "sum(container_threads{namespace=\"$namespace\", pod=\"$pod\", container!=\"POD\"})", + "interval": "", + "intervalFactor": 5, + "legendFormat": "Thread", + "range": true, + "refId": "A" + } + ], + "title": "Number of Thread", + "type": "timeseries" + } + ], + "preload": false, + "refresh": "30s", + "schemaVersion": 40, + "tags": [ + "kubernetes", + "k8s" + ], + "templating": { + "list": [ + { + "auto": false, + "auto_count": 30, + "auto_min": "10s", + "current": { + "text": "5m", + "value": "5m" + }, + "name": "duration", + "options": [ + { + "selected": false, + "text": "2m", + "value": "2m" + }, + { + "selected": true, + "text": "5m", + "value": "5m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + } + ], + "query": "2m,5m,10m,30m,1h", + "refresh": 2, + "type": "interval" + }, + { + "current": { + "text": "default", + "value": "default" + }, + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "definition": "label_values(kube_pod_info,namespace)", + "includeAll": false, + "name": "namespace", + "options": [], + "query": { + "query": "label_values(kube_pod_info,namespace)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "", + "sort": 1, + "type": "query" + }, + { + "current": { + "text": "db-54ffbcff85-6kzdn", + "value": "db-54ffbcff85-6kzdn" + }, + "datasource": { + "type": "prometheus", + "uid": "defp1aue65zb4b" + }, + "definition": "label_values(kube_pod_info{namespace=~\"$namespace\"},pod)", + "includeAll": false, + "name": "pod", + "options": [], + "query": { + "query": "label_values(kube_pod_info{namespace=~\"$namespace\"},pod)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "", + "sort": 5, + "type": "query" + } + ] + }, + "time": { + "from": "now-30m", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ] + }, + "timezone": "", + "title": "Kubernetes Pod Dashboard", + "uid": "4b545447f", + "version": 1, + "weekStart": "" + } \ No newline at end of file diff --git a/kubernetes/db-deploy-svc.yaml b/kubernetes/db-deploy-svc.yaml index fdf866f..872e673 100644 --- a/kubernetes/db-deploy-svc.yaml +++ b/kubernetes/db-deploy-svc.yaml @@ -51,7 +51,7 @@ spec: apiVersion: v1 kind: Service metadata: - name: db + name: db-service spec: selector: app: db diff --git a/kubernetes/db-pv.yaml b/kubernetes/db-pv.yaml new file mode 100644 index 0000000..16427cc --- /dev/null +++ b/kubernetes/db-pv.yaml @@ -0,0 +1,14 @@ +apiVersion: v1 +kind: PersistentVolume +metadata: + name: db-pv + labels: + type: local +spec: + capacity: + storage: 1Gi + accessModes: + - ReadWriteOnce + persistentVolumeReclaimPolicy: Retain + hostPath: + path: "/mnt/data/db" diff --git a/kubernetes/db-pvc.yaml b/kubernetes/db-pvc.yaml index 31dd758..73cb6ad 100644 --- a/kubernetes/db-pvc.yaml +++ b/kubernetes/db-pvc.yaml @@ -1,8 +1,6 @@ apiVersion: v1 kind: PersistentVolumeClaim metadata: - labels: - usage: db-data name: db-data spec: accessModes: @@ -10,3 +8,6 @@ spec: resources: requests: storage: 1Gi + selector: + matchLabels: + type: local diff --git a/kubernetes/network-policy.yaml b/kubernetes/network-policy.yaml index 1deafe9..fb1eca4 100644 --- a/kubernetes/network-policy.yaml +++ b/kubernetes/network-policy.yaml @@ -13,6 +13,9 @@ spec: - podSelector: matchLabels: app: api + - podSelector: + matchLabels: + app.kubernetes.io/name: grafana ports: - protocol: TCP port: 5432 \ No newline at end of file diff --git a/project-info/api-logs-dashboard.png b/project-info/api-logs-dashboard.png new file mode 100644 index 0000000..75368dc Binary files /dev/null and b/project-info/api-logs-dashboard.png differ diff --git a/project-info/api-pod-dashboard.png b/project-info/api-pod-dashboard.png new file mode 100644 index 0000000..ad9be7d Binary files /dev/null and b/project-info/api-pod-dashboard.png differ diff --git a/project-info/api.svg b/project-info/api.svg new file mode 100644 index 0000000..8cf1524 --- /dev/null +++ b/project-info/api.svg @@ -0,0 +1,2 @@ +
POST
CreateUser
POST...
name string

Return User
name string...
/v1
/v1
API
<ip-address>:8080/
API...
GET
GetUser
GET...
Return User
Return User
POST
CreateFeed
POST...
name string
url string

Return Feed
name string...
Entity: User 
Path: /users
Entity: User...
user_id UUID
created_at TIMESTAMP
updated_at TIMESTAMP
name TEXT
api_key VARCHAR(64)
user_id UUID...
Entity: Feed
Path: /feeds
Entity: Feed...
feed_id UUID
created_at TIMESTAMP
updated_at TIMESTAMP
name TEXT
url TEXT
user_id UUID
feed_id UUID...
GET
GetFeeds
GET...
Return []Feed
Return []Feed
Entity: Post
Path: /posts
Entity: Post...
post_id UUID
created_at TIMESTAMP
updated_at TIMESTAMP
title TEXT
summary TEXT
content TEXT
url TEXT
published_at TIMESTAMP
feed_id UUID
post_id UUID...
GET
GetPostsForUser
GET...
limit int
offset int

Return []Post
limit int...
Entity: FeedFollows Path: /feed_follows
Entity: FeedFollows Pa...
feed_follow_id UUID
created_at TIMESTAMP
updated_at TIMESTAMP
user_id UUID
feed_id UUID
feed_follow_id UUID...
POST
CreateFeedFollow
POST...
feed_id string

Return FeedFollow
feed_id string...
GET
GetFeedFollows
GET...
Return []FeedFollow
Return []FeedFollow
DELETE
DeleteFeedFollow
DELETE...
feed_follow_id string
feed_follow_id string
Requires Authentication (user's API key)
Requires Authenticatio...
Middleware to collect
and store request/response logs
Middleware to collect...Text is not SVG - cannot display \ No newline at end of file diff --git a/project-info/cicd-flow.svg b/project-info/cicd-flow.svg new file mode 100644 index 0000000..c54f750 --- /dev/null +++ b/project-info/cicd-flow.svg @@ -0,0 +1 @@ +
Lint & Test & Push
Lint & Test & PushAPIText is not SVG - cannot display \ No newline at end of file diff --git a/project-info/db-pod-dashboard.png b/project-info/db-pod-dashboard.png new file mode 100644 index 0000000..29ae14f Binary files /dev/null and b/project-info/db-pod-dashboard.png differ diff --git a/project-info/k8s-all.png b/project-info/k8s-all.png new file mode 100644 index 0000000..1790f89 Binary files /dev/null and b/project-info/k8s-all.png differ diff --git a/project-info/k8s-flow.svg b/project-info/k8s-flow.svg new file mode 100644 index 0000000..bb2fd33 --- /dev/null +++ b/project-info/k8s-flow.svg @@ -0,0 +1 @@ +
API Traffic
API Trafficdeploypod
api
apipvpvcsvc
api-service
api-servicesecretsvc
db-service
db-servicedeploypod
db
dbnetpol
db-access-policy
db-acce...node
local-kind-ctrl-plane
local-kind-...
Monitoring & Metrics
Monitoring & MetricsText is not SVG - cannot display \ No newline at end of file diff --git a/sqlc/schema/001_users.sql b/sqlc/schema/001_users.sql index a309225..014de85 100644 --- a/sqlc/schema/001_users.sql +++ b/sqlc/schema/001_users.sql @@ -4,7 +4,7 @@ create table users ( id uuid primary key, created_at timestamp not null, updated_at timestamp not null, - name text not null + name text unique not null ); -- +goose Down