From 7ff55d7a9f574c942836cdc572eb9758bf0f945c Mon Sep 17 00:00:00 2001 From: Dmitriy Creed Date: Thu, 15 May 2025 20:58:29 +0300 Subject: [PATCH 01/16] Update README.md --- README.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 0b159ed716..e0bbd103fb 100644 --- a/README.md +++ b/README.md @@ -6,22 +6,22 @@ Welcome to the DevOps Engineering course labs! These hands-on labs are designed ## Lab Syllabus -Lab 1: Web Application Development -Lab 2: Containerization -Lab 3: Continuous Integration -Lab 4: Infrastructure as Code & Terraform -Lab 5: Configuration Management -Lab 6: Ansible Automation -Lab 7: Observability, Logging, Loki Stack -Lab 8: Monitoring & Prometheus -Lab 9: Kubernetes & Declarative Manifests -Lab 10: Helm Charts & Library Charts -Lab 11: Kubernetes Secrets Management (Vault, ConfigMaps) -Lab 12: Kubernetes ConfigMaps & Environment Variables -Lab 13: GitOps with ArgoCD -Lab 14: StatefulSet Optimization -Lab 15: Kubernetes Monitoring & Init Containers -Lab 16: IPFS & Fleek Decentralization +1. Web Application Development +2. Containerization +3. Continuous Integration +4. Infrastructure as Code & Terraform +5. Configuration Management +6. Ansible Automation +7. Observability, Logging, Loki Stack +8. Monitoring & Prometheus +9. Kubernetes & Declarative Manifests +10. Helm Charts & Library Charts +11. Kubernetes Secrets Management (Vault, ConfigMaps) +12. Kubernetes ConfigMaps & Environment Variables +13. GitOps with ArgoCD +14. StatefulSet Optimization +15. Kubernetes Monitoring & Init Containers +16. IPFS & Fleek Decentralization ## Architecture From a057c02a3a451871d1c12ae9a7d7c4a781c0bec6 Mon Sep 17 00:00:00 2001 From: Dmitrii Creed Date: Wed, 7 Jan 2026 23:30:57 +0400 Subject: [PATCH 02/16] Publish labs 01-06 Signed-off-by: Dmitrii Creed --- README.md | 1343 +++++++++++++++++++++++++++++++++++++++-- labs/lab01.md | 689 +++++++++++++++++++++ labs/lab02.md | 362 +++++++++++ labs/lab03.md | 928 ++++++++++++++++++++++++++++ labs/lab04.md | 1508 ++++++++++++++++++++++++++++++++++++++++++++++ labs/lab05.md | 974 ++++++++++++++++++++++++++++++ labs/lab06.md | 1592 +++++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 7339 insertions(+), 57 deletions(-) create mode 100644 labs/lab01.md create mode 100644 labs/lab02.md create mode 100644 labs/lab03.md create mode 100644 labs/lab04.md create mode 100644 labs/lab05.md create mode 100644 labs/lab06.md diff --git a/README.md b/README.md index e0bbd103fb..d542ddc486 100644 --- a/README.md +++ b/README.md @@ -1,81 +1,1310 @@ -# DevOps Engineering Labs +# 🚀 DevOps Engineering: Core Practices & Advanced Infrastructure -## Introduction +[![Labs](https://img.shields.io/badge/Labs-70%25-blue)](#-lab-based-learning-experience) +[![Exam](https://img.shields.io/badge/Exam-30%25-orange)](#-evaluation-framework) +[![Hands-On](https://img.shields.io/badge/Focus-Production--Ready%20Skills-success)](#-lab-based-learning-experience) +[![Duration](https://img.shields.io/badge/Duration-14%20Weeks-lightgrey)](#-course-roadmap) -Welcome to the DevOps Engineering course labs! These hands-on labs are designed to guide you through various aspects of DevOps practices and principles. As you progress through the labs, you'll gain practical experience in application development, containerization, testing, infrastructure setup, CI/CD processes, and more. +Welcome to the **DevOps Engineering Core Course**, where you will master **production-grade DevOps practices** through hands-on implementation. +This comprehensive course takes you from application development to advanced Kubernetes orchestration and decentralized infrastructure. -## Lab Syllabus +Through **progressive labs and real-world scenarios**, you'll build, containerize, deploy, monitor, and scale applications — mastering the complete DevOps lifecycle used by industry-leading teams. -1. Web Application Development -2. Containerization -3. Continuous Integration -4. Infrastructure as Code & Terraform -5. Configuration Management -6. Ansible Automation -7. Observability, Logging, Loki Stack -8. Monitoring & Prometheus -9. Kubernetes & Declarative Manifests -10. Helm Charts & Library Charts -11. Kubernetes Secrets Management (Vault, ConfigMaps) -12. Kubernetes ConfigMaps & Environment Variables -13. GitOps with ArgoCD -14. StatefulSet Optimization -15. Kubernetes Monitoring & Init Containers -16. IPFS & Fleek Decentralization +--- -## Architecture +## 📚 Course Roadmap -This repository has a master branch containing an introduction. Each new lab assignment will be added as a markdown file with a lab number. +**Intensive course** with progressive modules building production-ready DevOps expertise: -## Rules +| Lab | Module | Key Topics & Technologies | +|-----|-------------------------------------------|--------------------------------------------------------------------------------------------------------------------------| +| 1 | **Web Application Development** | Python/Go web apps, Moscow timezone handling, framework selection, best practices | +| 2 | **Containerization with Docker** | Dockerfiles, multi-stage builds, rootless containers, distroless images, Docker Hub | +| 3 | **Continuous Integration** | GitHub Actions, unit testing, linting, Snyk vulnerability scanning, Docker CI/CD | +| 4 | **Infrastructure as Code** | Terraform workflows, Docker infrastructure, Yandex Cloud/AWS, GitHub provider, state management | +| 5 | **Configuration Management - Ansible** | Ansible roles, playbooks, Docker deployment, dynamic inventory, structured automation | +| 6 | **Continuous Deployment - Ansible** | Application deployment roles, Docker Compose templates, tags, blocks, role dependencies | +| 7 | **Observability & Logging** | Promtail, Loki, Grafana stack, log aggregation, Docker Compose orchestration | +| 8 | **Monitoring with Prometheus** | Prometheus setup, metrics collection, Grafana dashboards, service health checks, resource limits | +| 9 | **Kubernetes Fundamentals** | Minikube, kubectl, Deployments, Services, declarative manifests, Ingress | +| 10 | **Helm Package Management** | Helm charts, chart hooks, library charts, templating, values management | +| 11 | **Kubernetes Secrets & Vault** | Kubernetes Secrets, HashiCorp Vault integration, sidecar injection, resource management | +| 12 | **ConfigMaps & Persistence** | ConfigMaps, persistent volumes, file mounting, environment variables, application state | +| 13 | **GitOps with ArgoCD** | ArgoCD setup, application sync, multi-environment deployment, auto-sync, self-healing | +| 14 | **StatefulSets & Optimization** | StatefulSet controllers, headless services, persistent storage, ordering guarantees, update strategies | +| 15 | **Kubernetes Monitoring & Init Containers** | Kube-Prometheus stack, Grafana dashboards, Init containers, queue patterns, cluster observability | +| 16 | **🎁 Decentralized Infrastructure** | IPFS gateway setup, Fleek deployment, decentralized storage, content addressing | -To successfully complete the labs and pass the course, follow these rules: +> **Note:** Lab 16 is a **bonus lab** for exploring cutting-edge decentralized technologies. Complete it for extra credit! -1. **Lab Dependency:** Complete the labs in order; each lab builds upon the previous one. -2. **Submission and Grading:** Submit your solutions as pull requests (PRs) to the master branch of this repository. You need at least 6/10 points for each lab to pass. -3. **Fork Repository:** Fork this repository to your workspace to create your own version for solving the labs. -4. **Recommended Workflow:** Build your solutions incrementally. Complete lab N based on lab N-1. -5. **PR Creation:** Create a PR from your fork to the master branch of this repository and from your fork's branch to your fork's master branch. -6. **Wait for Grade:** Once your PR is created, wait for your lab to be reviewed and graded. +--- -### Example for the first lab +## 🗺️ DevOps Learning Journey -1. Fork this repository. -2. Checkout to the lab1 branch. -3. Complete the lab1 tasks. -4. Push the code to your repository. -5. Create a PR to the master branch of this repository from your fork's lab1 branch. -6. Create a PR to the master branch of your repository from your lab1 branch. -7. Wait for your grade. +
+🌳 View Complete Skill Tree Structure -## Grading and Grades Distribution +```mermaid +graph LR + ROOT[🚀 DevOps Engineering Mastery] -Your final grade will be determined based on labs and a final exam: + %% Foundation Branch + ROOT --- FOUND[🏗️ Foundation Layer] + FOUND --- A[📱 Lab 1: Web Apps
• Python/Go Development
• Framework Selection
• Best Practices] + FOUND --- B[🐳 Lab 2: Containerization
• Docker Images
• Multi-stage Builds
• Distroless] -- Labs: 70% of your final grade. -- Final Exam: 30% of your final grade. + %% CI/CD Branch + ROOT --- CICD[⚙️ CI/CD Pipeline] + CICD --- C[🔄 Lab 3: CI/CD
• GitHub Actions
• Testing & Linting
• Security Scanning] + CICD --- D[🏗️ Lab 4: Infrastructure
• Terraform
• Cloud Provisioning
• State Management] -Grade ranges: + %% Configuration Management Branch + ROOT --- CONFIG[🛠️ Configuration Management] + CONFIG --- E[📋 Lab 5: Ansible Basics
• Playbooks & Roles
• Docker Deployment
• Dynamic Inventory] + CONFIG --- F[🚀 Lab 6: CD with Ansible
• App Deployment
• Templates
• Role Dependencies] -- [90-100] - A -- [75-90) - B -- [60-75) - C -- [0-60) - D + %% Observability Branch + ROOT --- OBS[📊 Observability Stack] + OBS --- G[📝 Lab 7: Logging
• Loki Stack
• Log Aggregation
• Grafana] + OBS --- H[📈 Lab 8: Monitoring
• Prometheus
• Metrics Collection
• Dashboards] -### Labs Grading + %% Kubernetes Foundation + ROOT --- K8S[☸️ Kubernetes Ecosystem] + K8S --- I[⚓ Lab 9: K8s Basics
• Minikube
• Deployments
• Services & Ingress] + K8S --- J[📦 Lab 10: Helm
• Chart Creation
• Hooks
• Library Charts] -Each lab is worth 10 points. Completing main tasks correctly earns you 10 points. Completing bonus tasks correctly adds 2.5 points. You can earn a maximum of 12.5 points per lab by completing all main and bonus tasks. + %% Advanced Kubernetes + ROOT --- K8SADV[🎯 Advanced K8s] + K8SADV --- K[🔐 Lab 11: Secrets
• Vault Integration
• Secret Management
• Resources] + K8SADV --- L[⚙️ Lab 12: ConfigMaps
• Configuration
• Persistence
• Volume Mounts] -Finishing all bonus tasks lets you skip the exam and grants you 5 extra points. Incomplete bonus tasks require you to take the exam, which could save you from failing it. + %% Production Kubernetes + ROOT --- PROD[🌐 Production K8s] + PROD --- M[🔄 Lab 13: GitOps
• ArgoCD
• Multi-env Deploy
• Auto-sync] + PROD --- N[💾 Lab 14: StatefulSets
• Stateful Apps
• Persistent Storage
• Update Strategies] ->The labs account for 70% of your final grade. With 14 labs in total, each lab contributes 5% to your final grade. Completing all main tasks in a lab earns you the maximum 10 points, which corresponds to 5% of your final grade. ->If you successfully complete all bonus tasks, you'll earn an additional 2.5 points, totaling 12.5 points for that lab, or 6.25% of your final grade. Over the course of all 14 labs, the cumulative points from bonus tasks add up to 87.5% of your final grade. ->Additionally, a 5% bonus is granted for successfully finishing all bonus tasks, ensuring that if you successfully complete everything, your final grade will be 92.5%, which corresponds to an A grade. + %% Advanced Topics + ROOT --- ADV[⭐ Advanced Topics] + ADV --- O[📊 Lab 15: Cluster Monitoring
• Kube-Prometheus
• Init Containers
• Observability] + ADV --- P[🌐 Lab 16: Web3 Infra
• IPFS
• Fleek
• Decentralization] -## Deadlines and Labs Distribution + %% Styling + classDef rootStyle fill:#1a1a1a,stroke:#ffffff,stroke-width:3px,color:#ffffff + classDef branchStyle fill:#2c3e50,stroke:#3498db,stroke-width:2px,color:#ffffff + classDef foundationModule fill:#e8f8f5,stroke:#16a085,stroke-width:2px,color:#2c3e50 + classDef cicdModule fill:#fdf2e9,stroke:#e67e22,stroke-width:2px,color:#2c3e50 + classDef configModule fill:#eaf2f8,stroke:#3498db,stroke-width:2px,color:#2c3e50 + classDef obsModule fill:#f4ecf7,stroke:#9b59b6,stroke-width:2px,color:#2c3e50 + classDef k8sModule fill:#e8f6f3,stroke:#1abc9c,stroke-width:2px,color:#2c3e50 + classDef prodModule fill:#fdedec,stroke:#e74c3c,stroke-width:2px,color:#2c3e50 + classDef advModule fill:#fff3cd,stroke:#f1c40f,stroke-width:2px,color:#2c3e50 -Each week, two new labs will be available. You'll have one week to submit your solutions. Refer to Moodle for presentation slides and deadlines. + class ROOT rootStyle + class FOUND,CICD,CONFIG,OBS,K8S,K8SADV,PROD,ADV branchStyle + class A,B foundationModule + class C,D cicdModule + class E,F configModule + class G,H obsModule + class I,J,K,L k8sModule + class M,N prodModule + class O,P advModule +``` -## Submission Policy +
-Submitting your lab results on time is crucial for your grading. Late submissions receive a maximum score of 6 points for the corresponding lab. Remember, completing all labs is necessary to successfully pass the course. +
+🏗️ View Technology Stack Progression + +```mermaid +flowchart TD + subgraph "⭐ Advanced & Emerging" + direction LR + O[📊 Lab 15: Cluster Monitoring
Kube-Prometheus & Init Containers] + P[🌐 Lab 16: IPFS & Fleek
Decentralized Infrastructure] + end + + subgraph "🌐 Production Kubernetes" + direction LR + M[🔄 Lab 13: ArgoCD
GitOps Workflows] + N[💾 Lab 14: StatefulSets
Stateful Applications] + end + + subgraph "🎯 Advanced Kubernetes" + direction LR + K[🔐 Lab 11: Vault
Secret Management] + L[⚙️ Lab 12: ConfigMaps
Configuration & Persistence] + end + + subgraph "☸️ Kubernetes Foundation" + direction LR + I[⚓ Lab 9: K8s Basics
Deployments & Services] + J[📦 Lab 10: Helm
Package Management] + end + + subgraph "📊 Observability Stack" + direction LR + G[📝 Lab 7: Loki
Logging Stack] + H[📈 Lab 8: Prometheus
Metrics & Monitoring] + end + + subgraph "🛠️ Configuration Management" + direction LR + E[📋 Lab 5: Ansible
Automation Basics] + F[🚀 Lab 6: CD
Deployment Automation] + end + + subgraph "⚙️ CI/CD & Infrastructure" + direction LR + C[🔄 Lab 3: GitHub Actions
Continuous Integration] + D[🏗️ Lab 4: Terraform
Infrastructure as Code] + end + + subgraph "🏗️ Foundation Layer" + direction LR + A[📱 Lab 1: Web Apps
Application Development] + B[🐳 Lab 2: Docker
Containerization] + end + + A --> B + B --> C + B --> D + C --> E + D --> E + E --> F + F --> G + F --> H + G --> I + H --> I + I --> J + J --> K + J --> L + K --> M + L --> M + M --> N + N --> O + O -.-> P + + classDef foundation fill:#e8f6f3,stroke:#1abc9c,stroke-width:3px,color:#2c3e50 + classDef cicd fill:#fef9e7,stroke:#f39c12,stroke-width:3px,color:#2c3e50 + classDef config fill:#eaf2f8,stroke:#3498db,stroke-width:3px,color:#2c3e50 + classDef observability fill:#f4ecf7,stroke:#9b59b6,stroke-width:3px,color:#2c3e50 + classDef kubernetes fill:#e8f8f5,stroke:#16a085,stroke-width:3px,color:#2c3e50 + classDef production fill:#fdedec,stroke:#e74c3c,stroke-width:3px,color:#2c3e50 + classDef advanced fill:#fff3cd,stroke:#f1c40f,stroke-width:3px,color:#2c3e50 + + class A,B foundation + class C,D cicd + class E,F config + class G,H observability + class I,J,K,L kubernetes + class M,N production + class O,P advanced +``` + +
+ +--- + +## 🛠 Lab-Based Learning Experience + +**Your grade comes from progressive labs and a comprehensive final exam** — building real-world DevOps expertise. + +> **Key Points:** +> - **16 progressive labs** covering the complete DevOps lifecycle +> - **15 required labs** provide core competencies +> - **Lab 16 is optional** for extra credit (up to 2.5 bonus points) +> - **Each lab builds on previous knowledge** — follow the sequence +> - **Minimum 6/10 points per lab** required to pass + +### Lab Structure + +* **Progressive Curriculum:** Start with app development, progress to Kubernetes orchestration +* **Production Focus:** Real-world tools and workflows used by industry teams +* **Hands-On Implementation:** Build, deploy, monitor, and scale applications +* **Best Practices:** Security, observability, automation, and GitOps principles +* **Comprehensive Documentation:** Technical writing and analysis skills +* **Bonus Opportunities:** Extra credit tasks in each lab (2.5 points each) + +### Lab Overview by Category + +
+📋 View Complete Lab Breakdown + +**🏗️ Foundation & Development (Labs 1-2):** + +1. **Web Application Development** — Python/Go web apps with best practices + - Framework selection and justification + - Moscow timezone implementation + - Clean code and testing practices + - **Bonus:** Additional language implementation (2.5 pts) + +2. **Containerization with Docker** — Production-ready containerization + - Dockerfile best practices and linting + - Rootless containers and security + - Multi-stage builds optimization + - **Bonus:** Distroless images comparison (2.5 pts) + +**⚙️ CI/CD & Infrastructure (Labs 3-4):** + +3. **Continuous Integration** — Automated testing and deployment + - GitHub Actions workflows + - Unit testing and linting + - Docker build and push automation + - Snyk vulnerability scanning + - **Bonus:** Multi-app CI workflows (2.5 pts) + +4. **Infrastructure as Code with Terraform** — Cloud provisioning automation + - Docker infrastructure with Terraform + - Yandex Cloud/AWS VM provisioning + - GitHub repository management + - State management and best practices + - **Bonus:** GitHub teams management (2.5 pts) + +**🛠️ Configuration Management (Labs 5-6):** + +5. **Ansible & Docker Deployment** — Configuration automation + - Custom Ansible roles for Docker + - Playbook development and testing + - Dynamic inventory setup + - Structured repository organization + - **Bonus:** Dynamic inventory & security (2.5 pts) + +6. **Continuous Deployment with Ansible** — Application deployment automation + - Application deployment roles + - Docker Compose templates (Jinja2) + - Role dependencies and tags + - Wipe logic and idempotency + - **Bonus:** Multi-app playbooks (2.5 pts) + +**📊 Observability Stack (Labs 7-8):** + +7. **Monitoring & Logging with Loki** — Log aggregation and analysis + - Promtail, Loki, Grafana stack + - Docker Compose orchestration + - Log collection and visualization + - Component architecture understanding + - **Bonus:** Multi-container logging (2.5 pts) + +8. **Monitoring with Prometheus** — Metrics collection and alerting + - Prometheus setup and configuration + - Grafana dashboard creation + - Service health monitoring + - Resource limits and log rotation + - **Bonus:** Application metrics & health checks (2.5 pts) + +**☸️ Kubernetes Foundation (Labs 9-10):** + +9. **Introduction to Kubernetes** — Container orchestration fundamentals + - Minikube cluster setup + - Deployments and Services + - Declarative manifest creation + - Ingress configuration + - **Bonus:** Multi-app deployments with Ingress (2.5 pts) + +10. **Helm Charts & Package Management** — Kubernetes application packaging + - Helm chart creation and customization + - Chart hooks (pre-install, post-install) + - Library charts for reusability + - Values management and templating + - **Bonus:** Library chart for multiple apps (2.5 pts) + +**🎯 Advanced Kubernetes (Labs 11-12):** + +11. **Kubernetes Secrets & HashiCorp Vault** — Secure secret management + - Kubernetes Secrets creation + - Vault installation via Helm + - Vault sidecar injection + - Kubernetes authentication + - **Bonus:** Resource management & named templates (2.5 pts) + +12. **ConfigMaps & Persistent Storage** — Configuration and data persistence + - ConfigMap creation and mounting + - Persistent volumes and claims + - Application state management + - File and environment variable injection + - **Bonus:** Multi-app persistence (2.5 pts) + +**🌐 Production Kubernetes (Labs 13-14):** + +13. **GitOps with ArgoCD** — Declarative continuous deployment + - ArgoCD installation and configuration + - Application sync automation + - Multi-environment deployment (dev/prod) + - Auto-sync and self-healing + - **Bonus:** Bonus app ArgoCD integration (2.5 pts) + +14. **StatefulSets & Optimization** — Managing stateful applications + - StatefulSet implementation + - Persistent storage validation + - Headless service configuration + - Parallel pod operations + - **Bonus:** Update strategies exploration (2.5 pts) + +**⭐ Advanced Topics (Labs 15-16):** + +15. **Kubernetes Monitoring & Init Containers** — Cluster observability + - Kube-Prometheus stack installation + - Grafana dashboard exploration + - Init container implementation + - Cluster metrics analysis + - **Bonus:** App metrics & init container queue (2.5 pts) + +16. **🎁 IPFS & Fleek Decentralization** — Web3 infrastructure (OPTIONAL) + - IPFS gateway setup with Docker + - File upload and verification + - Fleek deployment + - Decentralized storage concepts + +> **Bonus Lab Benefits:** +> - Lab 16 is completely optional but provides up to 2.5 bonus points +> - Explore cutting-edge decentralized technologies +> - Perfect for students interested in Web3 infrastructure +> - Counts toward your final grade as extra credit + +
+ +### Submission Workflow + +```mermaid +graph LR + A[Fork Repository] --> B[Create Lab Branch] + B --> C[Complete Lab Tasks] + C --> D[Test & Document] + D --> E[Push to Fork] + E --> F[Create PR to Main Repo] + F --> G[Create PR to Fork Master] + G --> H[Wait for Review] + H --> I[Receive Grade] + + style A fill:#e8f8f5,stroke:#16a085,color:#2c3e50 + style B fill:#e8f8f5,stroke:#16a085,color:#2c3e50 + style C fill:#fef9e7,stroke:#f39c12,color:#2c3e50 + style D fill:#fef9e7,stroke:#f39c12,color:#2c3e50 + style E fill:#eaf2f8,stroke:#3498db,color:#2c3e50 + style F fill:#f4ecf7,stroke:#9b59b6,color:#2c3e50 + style G fill:#f4ecf7,stroke:#9b59b6,color:#2c3e50 + style H fill:#fdedec,stroke:#e74c3c,color:#2c3e50 + style I fill:#e8f6f3,stroke:#1abc9c,color:#2c3e50 +``` + +
+📋 Detailed Submission Process + +**Step-by-Step Submission Guide:** + +1. **Fork the Course Repository** + - Navigate to the course repository on GitHub + - Click "Fork" to create your own copy + - Clone your fork locally: + ```bash + git clone https://github.com/YOUR_USERNAME/DevOps-Core-Course.git + cd DevOps-Core-Course + ``` + +2. **Create a Lab Branch** + - Always work on feature branches, never directly on master: + ```bash + git checkout -b lab1 + # or for subsequent labs + git checkout -b lab2 + ``` + +3. **Complete Lab Tasks** + - Follow the lab instructions in the `labN.md` file + - Create required folders and files + - Implement all main tasks (10 points) + - Attempt bonus tasks if possible (2.5 points) + +4. **Test & Document** + - Test all commands and verify outputs + - Take screenshots for proof of work + - Create required documentation files (.md) + - Ensure proper Markdown formatting + +5. **Commit and Push Changes** + - Stage and commit your work: + ```bash + git add . + git commit -m "Complete lab1: Web application development" + git push -u origin lab1 + ``` + +6. **Create Pull Requests (2 PRs Required)** + + **PR #1: Fork → Course Repository** + - Go to the **course repository** (original, not your fork) + - Click "New Pull Request" + - Click "compare across forks" + - Set: `base repository: course-repo` ← `head repository: your-fork` + - Set: `base: master` ← `compare: lab1` + - Title: `Lab 1 Submission - Your Name` + - Fill PR template with completion checklist + + **PR #2: Lab Branch → Your Fork's Master** + - Go to **your fork** + - Click "New Pull Request" + - Set: `base: master` ← `compare: lab1` + - Title: `Lab 1 - Web Application Development` + +7. **Wait for Review & Grade** + - Instructor will review both PRs + - Grade will be provided with feedback + - Address any required changes if needed + +**⚠️ Important Reminders:** +- Create **both PRs** as specified (fork→course AND lab→fork-master) +- Submit **before the deadline** (late submissions max 6/10 points) +- Use **proper Markdown formatting** ([check with linter](https://dlaa.me/markdownlint/)) +- Test **everything works** before submission +- Include **all required documentation** + +
+ +### Grading Policy + +
+🎯 Lab Grading Breakdown + +**Each Lab Scoring:** + +* **Main Tasks (10 points maximum):** + - **10/10:** All tasks completed perfectly + - Full implementation with correct functionality + - Comprehensive documentation + - Best practices applied + - Clean code and proper structure + - On-time submission + + - **8-9/10:** Strong submission with minor issues + - All tasks completed + - Good documentation + - Minor improvements needed + - Functional implementation + + - **6-7/10:** Passing submission (minimum to pass) + - Core tasks completed + - Basic documentation present + - Some areas need improvement + - Generally functional + + - **< 6/10:** Below passing + - Incomplete tasks + - Insufficient documentation + - Major functionality gaps + - Requires resubmission + +* **Bonus Tasks (2.5 points per lab):** + - Additional challenges beyond main requirements + - Same quality standards as main tasks + - Adds to overall grade (can exceed 10/10 per lab) + - No penalty for not completing + - Great for grade buffer + +**Late Submission Policy:** +- **On-time:** Full points possible (up to 12.5/10 with bonus) +- **Late (within 1 week):** Maximum 6/10 points + - Bonus tasks not graded for late submissions + - Must complete all main tasks + - 1-7 days after deadline +- **Very Late (> 1 week):** No credit accepted + +**Resubmission Policy:** +- One resubmission allowed if score < 6/10 +- Must address all feedback points +- Resubmission deadline: 1 week after grade received +- Maximum 8/10 on resubmissions + +
+ +--- + +## 📊 Evaluation Framework + +### Grade Composition + +| Component | Weight | Points | Details | +|-----------|--------|--------|---------| +| **Labs (15 required)** | 70% | 150 points | 15 labs × 10 points each | +| **Final Exam** | 30% | 50 points | Comprehensive assessment covering all labs | +| **Bonus Tasks** | Extra | +37.5 points max | 15 labs × 2.5 bonus points each | +| **Lab 16 (Optional)** | Bonus | +12.5 points | Optional IPFS/Fleek lab with 10 main + 2.5 bonus | +| **Total Base** | 100% | 200 points | Required to pass: 120+ points (60%) | +| **Maximum Possible** | — | 247.5 points | With all bonuses (capped at 100% for final grade) | + +### Performance Tiers + +
+📈 Grading Scale & Requirements + +| Grade | Range | Percentage | Description | +|-------|-------|------------|-------------| +| **A** | 180-200+ | 90-100% | Exceptional mastery, innovative solutions, comprehensive documentation | +| **B** | 150-179 | 75-89% | Strong understanding, consistent quality, minor improvements needed | +| **C** | 120-149 | 60-74% | Adequate competency, basic requirements met, needs reinforcement | +| **D** | 0-119 | 0-59% | Insufficient understanding, major gaps, re-attempt required | + +**Grade Calculation Examples:** + +**Scenario 1: Perfect Required Labs + Exam** +``` +Required Labs: 15 × 10 = 150 points +Final Exam: 60/60 points +Total: 200/200 = 100% (A) +``` + +**Scenario 2: Good Labs + Good Exam** +``` +Required Labs: 15 × 9 = 135 points (avg 9/10 per lab) +Final Exam: 45/60 points +Total: 180/200 = 90% (A) +``` + +**Scenario 3: All Labs Passed + Average Exam** +``` +Required Labs: 15 × 8 = 120 points (avg 8/10 per lab) +Final Exam: 37/60 points +Total: 157/200 = 78.5% (B) +``` + +**Scenario 4: With Bonus Tasks (Grade Buffer)** +``` +Required Labs: 15 × 9 = 135 points +Bonus Tasks: 10 labs × 2.5 = 25 points +Final Exam: 39/60 points +Total: 199/200 = 99.5% (A) +``` + +**Scenario 5: With Optional Lab 16** +``` +Required Labs: 15 × 9 = 135 points +Bonus Tasks: 8 labs × 2.5 = 20 points +Lab 16: 10 + 2.5 bonus = 12.5 points +Final Exam: 50/60 points +Total: 212.5 points → 100% (capped at A+) +``` + +**Minimum Passing Requirements:** +- Must complete **all 15 required labs** with at least 6/10 each +- Minimum 90 points from labs (15 × 6) +- Must take final exam (minimum 30/60 to reach 120 total) +- Total minimum: 120/200 points (60%) + +**⚠️ Failing Conditions:** +- Any lab with < 6/10 points (must resubmit) +- Total < 120 points after labs + exam +- Missing any required lab submission +- Academic dishonesty or plagiarism + +
+ +
+🏆 Bonus Points Strategy + +**Maximizing Your Grade with Bonuses:** + +**Bonus Task Opportunities (35 points maximum):** +- Lab 1: +2.5 (additional language web app) +- Lab 2: +2.5 (distroless images) +- Lab 3: +2.5 (multi-app CI) +- Lab 4: +2.5 (GitHub teams) +- Lab 5: +2.5 (dynamic inventory) +- Lab 6: +2.5 (multi-app playbooks) +- Lab 7: +2.5 (multi-container logging) +- Lab 8: +2.5 (app metrics) +- Lab 9: +2.5 (multi-app Ingress) +- Lab 10: +2.5 (library charts) +- Lab 11: +2.5 (resource management) +- Lab 12: +2.5 (multi-app persistence) +- Lab 13: +2.5 (bonus app ArgoCD) +- Lab 14: +2.5 (update strategies) +- Lab 15: +2.5 (metrics) + +**Optional Lab 16 (12.5 points):** +- Main tasks: 10 points +- Bonus task: 2.5 points +- Completely optional but recommended + +**Strategic Benefits:** +1. **Grade Insurance:** Bonus points provide buffer for exam performance +2. **Skill Development:** Bonus tasks explore advanced concepts +3. **Portfolio Building:** Extra implementations strengthen your portfolio +4. **Exam Flexibility:** High bonus points reduce exam pressure +5. **Learning Depth:** Go beyond basics to master tools + +**Recommended Approach:** +- **Priority 1:** Complete all required labs with 8+/10 +- **Priority 2:** Add bonus tasks to labs you found interesting +- **Priority 3:** Complete Lab 16 if interested in Web3/IPFS +- **Priority 4:** Focus on exam preparation with lab reviews + +**Example High-Achievement Path:** +``` +Required Labs: 15 × 10 = 150 points (perfect main tasks) +Bonus Tasks: 10 labs × 2.5 = 25 points (selective bonuses) +Lab 16: 12.5 points (optional) +Exam: 55/60 points (92%) +Total: 232.5 points → 100%+ (A+) +``` + +
+ +--- + +## ✅ Success Path + +> **"Complete all 15 required labs with minimum 6/10 each (84 pts). Score 36+ on final exam (60 pts). Total 120+ points = Pass. Aim for 180+ = A grade."** + +
+💡 Tips for Success + +**Lab Completion Strategy:** + +1. **Start Early** + - Begin each lab as soon as it's released + - Don't wait until deadline week + - Time management is critical with 15 labs + +2. **Read Instructions Thoroughly** + - Review entire lab document before starting + - Understand main vs bonus tasks + - Note all documentation requirements + - Check submission requirements + +3. **Follow the Progressive Path** + - Complete labs in sequence (1→16) + - Each lab builds on previous knowledge + - Don't skip ahead without completing prerequisites + - Review previous labs before starting new ones + +4. **Test Everything** + - Verify all commands work before documenting + - Test in clean environments when possible + - Take screenshots as proof of work + - Keep terminal outputs for documentation + +5. **Document as You Go** + - Don't leave documentation until the end + - Write explanations while fresh in mind + - Include both commands AND their purpose + - Explain your decisions and learnings + +**Time Allocation Guidelines:** + +| Lab Category | Estimated Time | Notes | +|--------------|----------------|-------| +| Labs 1-2 (Foundation) | 4-6 hours each | Setup time included | +| Labs 3-6 (CI/CD/Config) | 5-7 hours each | Learning new tools | +| Labs 7-8 (Observability) | 4-6 hours each | Mostly configuration | +| Labs 9-12 (Kubernetes) | 6-8 hours each | New paradigm, take time | +| Labs 13-15 (Advanced) | 5-7 hours each | Build on K8s knowledge | +| Lab 16 (Bonus) | 4-5 hours | Optional, Web3 concepts | +| Bonus Tasks | +1-2 hours each | Per bonus task | + +**Documentation Best Practices:** + +1. **Structure Your Documentation** + - Clear headings and sections + - Logical flow of information + - Code blocks for commands + - Screenshots for visual proof + +2. **Explain Your Thinking** + - Why you chose specific approaches + - What problems you encountered + - How you solved issues + - What you learned + +3. **Use Proper Markdown** + - Format with headers (##, ###) + - Code blocks with language hints + - Lists for step-by-step instructions + - Tables for comparisons + - Validate with [markdownlint](https://dlaa.me/markdownlint/) + +4. **Include All Requirements** + - Command outputs as specified + - Screenshots where requested + - Explanations of concepts + - Best practices you applied + +**Git Workflow Best Practices:** + +1. **Branch Strategy** + ```bash + # Always create branches from master + git checkout master + git pull origin master + git checkout -b lab3 + ``` + +2. **Commit Frequently** + ```bash + # Make small, logical commits + git add app_python/ + git commit -m "Add Python web application with timezone handling" + + git add app_python/PYTHON.md + git commit -m "Document Python app best practices" + ``` + +3. **Push Regularly** + ```bash + # Don't lose work - push often + git push -u origin lab3 + ``` + +4. **PR Quality** + - Use descriptive titles + - Fill out PR templates completely + - Reference lab number clearly + - Check that files render correctly on GitHub + +**Common Pitfalls to Avoid:** + +❌ **Don't:** +- Start labs late (deadline pressure causes errors) +- Copy-paste without understanding +- Skip documentation thinking you'll remember +- Work directly on master branch +- Submit without testing +- Ignore Markdown linting warnings +- Rush through bonus tasks for points +- Forget to create both required PRs + +✅ **Do:** +- Start early with time buffer +- Understand every command you run +- Document as you work +- Use proper Git workflow +- Test thoroughly before submission +- Validate Markdown formatting +- Only attempt bonuses if you have time +- Double-check PR destinations + +**Getting Help:** + +1. **Use Available Resources** + - Lab instructions and examples + - Official tool documentation + - Course slides and materials + - Previous lab solutions (your own) + +2. **Collaborate Appropriately** + - Discuss concepts with classmates + - Share understanding, not solutions + - Help others debug (don't give answers) + - Ask for clarification, not solutions + +3. **Ask Questions Early** + - Don't wait until deadline to ask + - Be specific about what you've tried + - Include error messages and context + - Show your debugging attempts + +4. **Office Hours & Forums** + - Attend office hours regularly + - Participate in course discussions + - Help answer others' questions (learn by teaching) + - Share useful resources you find + +
+ +
+📅 Recommended Study Schedule + +**Course Timeline:** + +**Weeks 1-8: Foundation & Core Skills** + +- **Week 1 - Lab 1:** Web Application Development + - Day 1-2: Python app development + - Day 3-4: Testing and documentation + - Day 5-6: Bonus task (optional second language) + - Day 7: Submit PRs + +- **Week 2 - Lab 2:** Containerization + - Day 1-2: Docker basics and Dockerfile creation + - Day 3-4: Implement best practices (rootless, layers) + - Day 5-6: Multi-stage builds and distroless (bonus) + - Day 7: Document and submit + +- **Week 3 - Lab 3:** Continuous Integration + - Day 1-2: GitHub Actions workflow creation + - Day 3-4: Testing, linting, Snyk integration + - Day 5-6: Docker CI/CD and multi-app workflows (bonus) + - Day 7: Finalize and submit + +- **Week 4 - Lab 4:** Infrastructure as Code + - Day 1-2: Terraform basics and Docker infrastructure + - Day 3-4: Cloud VM provisioning + - Day 5-6: GitHub provider and teams (bonus) + - Day 7: Documentation and submission + +- **Week 5 - Lab 5:** Ansible Basics + - Day 1-2: Ansible installation and role creation + - Day 3-4: Docker deployment playbooks + - Day 5-6: Dynamic inventory (bonus) + - Day 7: Testing and submission + +- **Week 6 - Lab 6:** Continuous Deployment + - Day 1-2: Application deployment role + - Day 3-4: Docker Compose templates and dependencies + - Day 5-6: Multi-app playbooks (bonus) + - Day 7: Finalize and submit + +- **Week 7 - Lab 7:** Logging Stack + - Day 1-2: Loki stack setup + - Day 3-4: Configuration and testing + - Day 5-6: Multi-container logging (bonus) + - Day 7: Documentation and submission + +- **Week 8 - Lab 8:** Prometheus Monitoring + - Day 1-2: Prometheus configuration + - Day 3-4: Grafana dashboards + - Day 5-6: Application metrics (bonus) + - Day 7: Review and submit + +**Weeks 9-14: Kubernetes Mastery** + +- **Week 9 - Lab 9:** Kubernetes Basics + - Day 1-2: Minikube setup and first deployment + - Day 3-4: Services and Ingress + - Day 5-6: Multi-app deployment (bonus) + - Day 7: Submit + +- **Week 10 - Lab 10:** Helm Charts + - Day 1-2: Chart creation and customization + - Day 3-4: Hooks implementation + - Day 5-6: Library charts (bonus) + - Day 7: Finalize and submit + +- **Week 11 - Lab 11:** Secrets & Vault + - Day 1-2: Kubernetes Secrets basics + - Day 3-4: Vault installation and integration + - Day 5-6: Resource management (bonus) + - Day 7: Document and submit + +- **Week 12 - Lab 12:** ConfigMaps & Persistence + - Day 1-2: ConfigMap creation and mounting + - Day 3-4: Persistent volumes + - Day 5-6: Multi-app persistence (bonus) + - Day 7: Submit + +- **Week 13 - Lab 13:** GitOps with ArgoCD + - Day 1-2: ArgoCD installation + - Day 3-4: Multi-environment deployment + - Day 5-6: Bonus app integration + - Day 7: Finalize and submit + +- **Week 14 - Lab 14:** StatefulSets + - Day 1-2: StatefulSet implementation + - Day 3-4: Storage validation and testing + - Day 5-6: Update strategies (bonus) + - Day 7: Submit + +**Post-Week 14: Advanced & Exam Prep** + +- **Week 15 - Lab 15:** Cluster Monitoring + - Complete within 1 week following same pattern + - Focus on Kube-Prometheus stack + - Init containers exploration + +- **Optional - Lab 16:** IPFS & Fleek + - Complete if time permits + - Great for resume and Web3 knowledge + - Extra credit toward final grade + +- **Exam Preparation:** + - Review all lab submissions + - Revisit key concepts from each module + - Practice commands and workflows + - Understand tool relationships + - Focus on "why" not just "what" + +**Daily Time Commitment:** +- Regular labs: 1-2 hours/day during lab weeks +- Kubernetes labs: 2-3 hours/day (more complex) +- Bonus tasks: +30-60 minutes if pursued +- Review time: 30 minutes/day between labs + +
+ +--- + +## 📚 Additional Resources + +
+🔗 Essential Documentation & Tools + +**Core Technologies:** + +**Version Control & Collaboration:** +- [Git Documentation](https://git-scm.com/doc) +- [GitHub Skills](https://skills.github.com/) +- [Pro Git Book](https://git-scm.com/book/en/v2) +- [GitHub Actions Documentation](https://docs.github.com/en/actions) + +**Containerization:** +- [Docker Documentation](https://docs.docker.com/) +- [Docker Best Practices](https://docs.docker.com/develop/dev-best-practices/) +- [Dockerfile Reference](https://docs.docker.com/engine/reference/builder/) +- [Docker Compose Documentation](https://docs.docker.com/compose/) + +**Infrastructure as Code:** +- [Terraform Documentation](https://www.terraform.io/docs) +- [Terraform Best Practices](https://www.terraform-best-practices.com/) +- [Terraform Registry](https://registry.terraform.io/) +- [Ansible Documentation](https://docs.ansible.com/) +- [Ansible Galaxy](https://galaxy.ansible.com/) + +**Observability:** +- [Grafana Documentation](https://grafana.com/docs/) +- [Loki Documentation](https://grafana.com/docs/loki/latest/) +- [Prometheus Documentation](https://prometheus.io/docs/) +- [Promtail Configuration](https://grafana.com/docs/loki/latest/clients/promtail/) + +**Kubernetes Ecosystem:** +- [Kubernetes Documentation](https://kubernetes.io/docs/) +- [Kubectl Reference](https://kubernetes.io/docs/reference/kubectl/) +- [Helm Documentation](https://helm.sh/docs/) +- [Minikube Documentation](https://minikube.sigs.k8s.io/docs/) +- [ArgoCD Documentation](https://argo-cd.readthedocs.io/) +- [HashiCorp Vault Documentation](https://www.vaultproject.io/docs) + +**Web3 & Decentralization (Lab 16):** +- [IPFS Documentation](https://docs.ipfs.tech/) +- [IPFS Concepts](https://docs.ipfs.tech/concepts/) +- [Fleek Documentation](https://docs.fleek.xyz/) +- [4EVERLAND Docs](https://docs.4everland.org/) + +**Security & Best Practices:** +- [OWASP Top 10](https://owasp.org/www-project-top-ten/) +- [Snyk Documentation](https://docs.snyk.io/) +- [CIS Docker Benchmark](https://www.cisecurity.org/benchmark/docker) +- [Kubernetes Security Best Practices](https://kubernetes.io/docs/concepts/security/security-best-practices/) + +
+ +
+📖 Learning Resources by Module + +**Foundation (Labs 1-2):** +- [Python Web Frameworks Comparison](https://www.fullstackpython.com/web-frameworks.html) +- [Go Web Development](https://golang.org/doc/articles/wiki/) +- [Docker Curriculum](https://docker-curriculum.com/) +- [Docker Security Best Practices](https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html) + +**CI/CD & IaC (Labs 3-4):** +- [GitHub Actions Best Practices](https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions) +- [Terraform Getting Started](https://learn.hashicorp.com/collections/terraform/aws-get-started) +- [Infrastructure as Code Principles](https://www.terraform.io/intro/index.html) + +**Configuration Management (Labs 5-6):** +- [Ansible for DevOps Book](https://www.ansiblefordevops.com/) +- [Ansible Best Practices](https://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html) +- [Jinja2 Templating](https://jinja.palletsprojects.com/en/3.0.x/templates/) + +**Observability (Labs 7-8):** +- [Observability Engineering Book](https://www.oreilly.com/library/view/observability-engineering/9781492076438/) +- [Grafana Fundamentals](https://grafana.com/tutorials/grafana-fundamentals/) +- [Prometheus Best Practices](https://prometheus.io/docs/practices/) + +**Kubernetes Foundation (Labs 9-10):** +- [Kubernetes Basics Interactive Tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) +- [Helm Chart Best Practices](https://helm.sh/docs/chart_best_practices/) +- [Learn Kubernetes Basics](https://kubernetes.io/docs/tutorials/) + +**Advanced Kubernetes (Labs 11-15):** +- [Kubernetes Patterns Book](https://k8spatterns.io/) +- [GitOps Principles](https://opengitops.dev/) +- [StatefulSet Best Practices](https://kubernetes.io/docs/tutorials/stateful-application/) +- [Kubernetes Security](https://kubernetes-security.info/) + +**Bonus & Advanced (Lab 16):** +- [IPFS Whitepaper](https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf) +- [Web3 Infrastructure Guide](https://web3.foundation/) +- [Decentralized Web Primer](https://flyingzumwalt.gitbooks.io/decentralized-web-primer/) + +
+ +
+🛠️ Required Tools & Installation + +**Core Tools (Required for Most Labs):** + +| Tool | Purpose | Installation | +|------|---------|--------------| +| **Git** | Version control | [Download](https://git-scm.com/downloads) | +| **Docker** | Containerization | [Download](https://docs.docker.com/get-docker/) | +| **Docker Compose** | Multi-container orchestration | [Download](https://docs.docker.com/compose/install/) | +| **VS Code / Editor** | Code editing | [Download](https://code.visualstudio.com/) | +| **Terminal/Shell** | Command line interface | Built-in (Linux/Mac) or [WSL2](https://docs.microsoft.com/en-us/windows/wsl/install) (Windows) | + +**Lab-Specific Tools:** + +**Labs 1-2 (Development & Containers):** +- Python 3.8+ or Go 1.16+ (depending on your choice) +- Docker Desktop or Docker Engine + +**Lab 3 (CI/CD):** +- GitHub account (free) +- Snyk account (free tier) + +**Lab 4 (Terraform):** +- [Terraform](https://www.terraform.io/downloads.html) (latest version) +- Yandex Cloud account (free tier) or AWS account +- VPN tool for cloud access + +**Labs 5-6 (Ansible):** +- [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) (2.9+) +- SSH access to a VM (from Lab 4) + +**Labs 7-8 (Observability):** +- Docker Compose (for stack deployment) +- Web browser for Grafana access + +**Labs 9-15 (Kubernetes):** +- [kubectl](https://kubernetes.io/docs/tasks/tools/) (latest stable) +- [Minikube](https://minikube.sigs.k8s.io/docs/start/) (v1.33.0 recommended) +- [Helm](https://helm.sh/docs/intro/install/) (v3.x) +- VirtualBox or Docker driver for Minikube + +**Lab 16 (IPFS - Optional):** +- Docker (for IPFS node) +- Fleek account (free) +- 4EVERLAND account (optional) + +**Platform-Specific Setup:** + +**Windows Users:** +- Install [WSL2](https://docs.microsoft.com/en-us/windows/wsl/install) (Ubuntu recommended) +- Install Docker Desktop with WSL2 backend +- Use Windows Terminal for better experience + +**macOS Users:** +- Install [Homebrew](https://brew.sh/) for package management +- Use built-in Terminal or iTerm2 + +**Linux Users:** +- Native Docker installation (not Docker Desktop) +- Package manager (apt/yum/dnf) for tools + +**Recommended VS Code Extensions:** +- Docker +- Kubernetes +- YAML +- Markdown All in One +- GitLens +- Terraform +- Ansible + +
+ +
+💡 Troubleshooting & Common Issues + +**Common Lab Issues & Solutions:** + +**Docker Issues:** +- **Problem:** Docker daemon not running + - **Solution:** `sudo systemctl start docker` (Linux) or start Docker Desktop +- **Problem:** Permission denied errors + - **Solution:** Add user to docker group: `sudo usermod -aG docker $USER` +- **Problem:** Port already in use + - **Solution:** Change port in configuration or stop conflicting service + +**Minikube Issues:** +- **Problem:** Minikube won't start + - **Solution:** Try different driver: `minikube start --driver=docker` +- **Problem:** Resource constraints + - **Solution:** Allocate more resources: `minikube start --memory=4096 --cpus=2` +- **Problem:** Version compatibility + - **Solution:** Use recommended versions from Lab 15 + +**Git & GitHub Issues:** +- **Problem:** PR to wrong repository + - **Solution:** Close PR and create new one to correct destination +- **Problem:** Large files in repository + - **Solution:** Use `.gitignore` and remove from history +- **Problem:** Merge conflicts + - **Solution:** Resolve manually or start from clean master + +**Terraform Issues:** +- **Problem:** State lock errors + - **Solution:** Force unlock: `terraform force-unlock ` +- **Problem:** Provider version conflicts + - **Solution:** Use version constraints in terraform block +- **Problem:** Authentication failures + - **Solution:** Check credentials and use environment variables + +**Ansible Issues:** +- **Problem:** SSH connection failures + - **Solution:** Verify SSH keys and host accessibility +- **Problem:** Module not found + - **Solution:** Install required collections: `ansible-galaxy collection install` +- **Problem:** Permission denied on target + - **Solution:** Check sudo configuration and privileges + +**Kubernetes Issues:** +- **Problem:** ImagePullBackOff errors + - **Solution:** Verify image name and registry access +- **Problem:** CrashLoopBackOff + - **Solution:** Check logs: `kubectl logs ` +- **Problem:** Service not accessible + - **Solution:** Check service type and use `minikube service` command + +**Getting More Help:** +1. Check lab-specific troubleshooting sections +2. Review tool documentation +3. Search for error messages (include tool name) +4. Ask in course forum with: + - Exact error message + - Commands you ran + - What you've already tried + - Relevant configuration files + +
+ +--- + +## 🎓 Course Completion + +Upon successful completion of this course, you will have: + +✅ **Full-Stack DevOps Expertise** — Complete pipeline from development to production +✅ **Production Portfolio** — 15+ documented projects with real infrastructure +✅ **Container Mastery** — Docker best practices and multi-stage optimization +✅ **CI/CD Implementation** — Automated testing, building, and deployment pipelines +✅ **Infrastructure as Code** — Terraform and Ansible for automated provisioning +✅ **Observability Skills** — Complete monitoring and logging stack implementation +✅ **Kubernetes Proficiency** — Container orchestration from basics to production patterns +✅ **GitOps Experience** — Declarative deployment with ArgoCD +✅ **Security Integration** — Secrets management with Vault and security scanning +✅ **Production Readiness** — StatefulSets, persistence, monitoring, and self-healing + +**Bonus Achievements (if completed):** +⭐ **Web3 Infrastructure** — Decentralized storage and hosting with IPFS/Fleek +⭐ **Advanced Kubernetes** — Library charts, init containers, and update strategies +⭐ **Multi-Language Experience** — Application development beyond Python + +--- + +## 📋 Quick Reference + +
+📌 Important Links & Commands + +**Course Links:** +- Course Repository: [GitHub Repository Link] +- Moodle Course: [Moodle Link] +- Lab Deadlines: Check Moodle +- Markdown Linter: https://dlaa.me/markdownlint/ + +**Essential Git Commands:** +```bash +# Setup +git clone https://github.com/YOUR_USERNAME/DevOps-Core-Course.git +git checkout -b labN + +# Development cycle +git status +git add . +git commit -m "descriptive message" +git push -u origin labN + +# Keep fork updated +git remote add upstream https://github.com/COURSE-ORG/DevOps-Core-Course.git +git fetch upstream +git merge upstream/master +``` + +**Docker Quick Reference:** +```bash +# Build and run +docker build -t myapp:latest . +docker run -p 8080:8080 myapp:latest + +# Manage containers +docker ps +docker logs +docker exec -it /bin/sh +docker stop +docker rm + +# Image management +docker images +docker rmi +docker push username/image:tag +``` + +**Kubernetes Quick Reference:** +```bash +# Cluster management +minikube start --driver=docker +minikube status +minikube service +minikube dashboard + +# Resource management +kubectl get pods,svc,deployments +kubectl describe pod +kubectl logs +kubectl exec -it -- /bin/sh +kubectl delete pod + +# Configuration +kubectl apply -f manifest.yaml +kubectl delete -f manifest.yaml +kubectl get events --sort-by='.lastTimestamp' +``` + +**Helm Quick Reference:** +```bash +# Chart management +helm create mychart +helm install myrelease mychart/ +helm list +helm upgrade myrelease mychart/ +helm rollback myrelease 1 +helm uninstall myrelease + +# Debugging +helm lint mychart/ +helm template myrelease mychart/ +helm install --dry-run --debug myrelease mychart/ +``` + +**Useful Debugging Commands:** +```bash +# Check system resources +docker stats +kubectl top nodes +kubectl top pods + +# Network debugging +curl -v http://localhost:8080 +nc -zv hostname port +nslookup domain + +# File system +ls -lah +du -sh * +df -h + +# Process management +ps aux | grep processname +systemctl status servicename +journalctl -u servicename -f +``` + +
+ +--- + +**Ready to begin your DevOps journey? Fork the repository and start with Lab 1!** 🚀 + +> **Remember:** Success in DevOps comes from hands-on practice, continuous learning, and understanding the "why" behind each tool and practice. Take your time, experiment, break things, and learn from failures — that's the DevOps way! + +**Questions?** Check the course Moodle page or ask during office hours. + +**Good luck!** 🎓 diff --git a/labs/lab01.md b/labs/lab01.md new file mode 100644 index 0000000000..2e53cb1ca2 --- /dev/null +++ b/labs/lab01.md @@ -0,0 +1,689 @@ +# Lab 1 — DevOps Info Service: Web Application Development + +![difficulty](https://img.shields.io/badge/difficulty-beginner-success) +![topic](https://img.shields.io/badge/topic-Web%20Development-blue) +![points](https://img.shields.io/badge/points-10%2B2.5-orange) +![languages](https://img.shields.io/badge/languages-Python%20|%20Go-informational) + +> Build a DevOps info service that reports system information and health status. This service will evolve throughout the course into a comprehensive monitoring tool. + +## Overview + +Create a **DevOps Info Service** - a web application providing detailed information about itself and its runtime environment. This foundation will grow throughout the course as you add containerization, CI/CD, monitoring, and persistence. + +**What You'll Learn:** +- Web framework selection and implementation +- System introspection and API design +- Python best practices and documentation +- Foundation for future DevOps tooling + +--- + +## Tasks + +### Task 1 — Python Web Application (6 pts) + +Build a production-ready Python web service with comprehensive system information. + +#### 1.1 Project Structure + +Create this structure: + +``` +app_python/ +├── app.py # Main application +├── requirements.txt # Dependencies +├── .gitignore # Git ignore +├── README.md # App documentation +├── tests/ # Unit tests (Lab 3) +│ └── __init__.py +└── docs/ # Lab documentation + ├── LAB01.md # Your lab submission + └── screenshots/ # Proof of work + ├── 01-main-endpoint.png + ├── 02-health-check.png + └── 03-formatted-output.png +``` + +#### 1.2 Choose Web Framework + +Select and justify your choice: +- **Flask** - Lightweight, easy to learn +- **FastAPI** - Modern, async, auto-documentation +- **Django** - Full-featured, includes ORM + +Document your decision in `app_python/docs/LAB01.md`. + +#### 1.3 Implement Main Endpoint: `GET /` + +Return comprehensive service and system information: + +```json +{ + "service": { + "name": "devops-info-service", + "version": "1.0.0", + "description": "DevOps course info service", + "framework": "Flask" + }, + "system": { + "hostname": "my-laptop", + "platform": "Linux", + "platform_version": "Ubuntu 24.04", + "architecture": "x86_64", + "cpu_count": 8, + "python_version": "3.13.1" + }, + "runtime": { + "uptime_seconds": 3600, + "uptime_human": "1 hour, 0 minutes", + "current_time": "2026-01-07T14:30:00.000Z", + "timezone": "UTC" + }, + "request": { + "client_ip": "127.0.0.1", + "user_agent": "curl/7.81.0", + "method": "GET", + "path": "/" + }, + "endpoints": [ + {"path": "/", "method": "GET", "description": "Service information"}, + {"path": "/health", "method": "GET", "description": "Health check"} + ] +} +``` + +
+💡 Implementation Hints + +**Get System Information:** +```python +import platform +import socket +from datetime import datetime + +hostname = socket.gethostname() +platform_name = platform.system() +architecture = platform.machine() +python_version = platform.python_version() +``` + +**Calculate Uptime:** +```python +start_time = datetime.now() + +def get_uptime(): + delta = datetime.now() - start_time + seconds = int(delta.total_seconds()) + hours = seconds // 3600 + minutes = (seconds % 3600) // 60 + return { + 'seconds': seconds, + 'human': f"{hours} hours, {minutes} minutes" + } +``` + +**Request Information:** +```python +# Flask +request.remote_addr # Client IP +request.headers.get('User-Agent') # User agent +request.method # HTTP method +request.path # Request path + +# FastAPI +request.client.host +request.headers.get('user-agent') +request.method +request.url.path +``` + +
+ +#### 1.4 Implement Health Check: `GET /health` + +Simple health endpoint for monitoring: + +```json +{ + "status": "healthy", + "timestamp": "2024-01-15T14:30:00.000Z", + "uptime_seconds": 3600 +} +``` + +Return HTTP 200 for healthy status. This will be used for Kubernetes probes in Lab 9. + +
+💡 Implementation Hints + +```python +# Flask +@app.route('/health') +def health(): + return jsonify({ + 'status': 'healthy', + 'timestamp': datetime.now(timezone.utc).isoformat(), + 'uptime_seconds': get_uptime()['seconds'] + }) + +# FastAPI +@app.get("/health") +def health(): + return { + 'status': 'healthy', + 'timestamp': datetime.now(timezone.utc).isoformat(), + 'uptime_seconds': get_uptime()['seconds'] + } +``` + +
+ +#### 1.5 Configuration + +Make your app configurable via environment variables: + +```python +import os + +HOST = os.getenv('HOST', '0.0.0.0') +PORT = int(os.getenv('PORT', 5000)) +DEBUG = os.getenv('DEBUG', 'False').lower() == 'true' +``` + +**Test:** +```bash +python app.py # Default: 0.0.0.0:5000 +PORT=8080 python app.py # Custom port +HOST=127.0.0.1 PORT=3000 python app.py +``` + +--- + +### Task 2 — Documentation & Best Practices (4 pts) + +#### 2.1 Application README (`app_python/README.md`) + +Create user-facing documentation: + +**Required Sections:** +1. **Overview** - What the service does +2. **Prerequisites** - Python version, dependencies +3. **Installation** + ```bash + python -m venv venv + source venv/bin/activate + pip install -r requirements.txt + ``` +4. **Running the Application** + ```bash + python app.py + # Or with custom config + PORT=8080 python app.py + ``` +5. **API Endpoints** + - `GET /` - Service and system information + - `GET /health` - Health check +6. **Configuration** - Environment variables table + +#### 2.2 Best Practices + +Implement these in your code: + +**1. Clean Code Organization** +- Clear function names +- Proper imports grouping +- Comments only where needed +- Follow PEP 8 + +
+💡 Example Structure + +```python +""" +DevOps Info Service +Main application module +""" +import os +import socket +import platform +from datetime import datetime, timezone +from flask import Flask, jsonify, request + +app = Flask(__name__) + +# Configuration +HOST = os.getenv('HOST', '0.0.0.0') +PORT = int(os.getenv('PORT', 5000)) + +# Application start time +START_TIME = datetime.now(timezone.utc) + +def get_system_info(): + """Collect system information.""" + return { + 'hostname': socket.gethostname(), + 'platform': platform.system(), + 'architecture': platform.machine(), + 'python_version': platform.python_version() + } + +@app.route('/') +def index(): + """Main endpoint - service and system information.""" + # Implementation +``` + +
+ +**2. Error Handling** + +
+💡 Implementation + +```python +@app.errorhandler(404) +def not_found(error): + return jsonify({ + 'error': 'Not Found', + 'message': 'Endpoint does not exist' + }), 404 + +@app.errorhandler(500) +def internal_error(error): + return jsonify({ + 'error': 'Internal Server Error', + 'message': 'An unexpected error occurred' + }), 500 +``` + +
+ +**3. Logging** + +
+💡 Implementation + +```python +import logging + +logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' +) +logger = logging.getLogger(__name__) + +logger.info('Application starting...') +logger.debug(f'Request: {request.method} {request.path}') +``` + +
+ +**4. Dependencies (`requirements.txt`)** + +```txt +# Web Framework +Flask==3.0.0 +# or +fastapi==0.104.1 +uvicorn==0.24.0 +``` + +Pin exact versions for reproducibility. + +**5. Git Ignore (`.gitignore`)** + +```gitignore +# Python +__pycache__/ +*.py[cod] +venv/ +*.log + +# IDE +.vscode/ +.idea/ + +# OS +.DS_Store +``` + +#### 2.3 Lab Submission (`app_python/docs/LAB01.md`) + +Document your implementation: + +**Required Sections:** +1. **Framework Selection** + - Your choice and why + - Comparison table with alternatives +2. **Best Practices Applied** + - List practices with code examples + - Explain importance of each +3. **API Documentation** + - Request/response examples + - Testing commands +4. **Testing Evidence** + - Screenshots showing endpoints work + - Terminal output +5. **Challenges & Solutions** + - Problems encountered + - How you solved them + +**Required Screenshots:** +- Main endpoint showing complete JSON +- Health check response +- Formatted/pretty-printed output + +#### 2.4 GitHub Community Engagement + +**Objective:** Explore GitHub's social features that support collaboration and discovery. + +**Actions Required:** +1. **Star** the course repository +2. **Follow** your professor and TAs on GitHub: + - Professor: [@Cre-eD](https://github.com/Cre-eD) + - TA: [@marat-biriushev](https://github.com/marat-biriushev) + - TA: [@pierrepicaud](https://github.com/pierrepicaud) +3. **Follow** at least 3 classmates from the course + +**Document in LAB01.md:** + +Add a "GitHub Community" section (after Challenges & Solutions) with 1-2 sentences explaining: +- Why starring repositories matters in open source +- How following developers helps in team projects and professional growth + +
+💡 GitHub Social Features + +**Why Stars Matter:** + +**Discovery & Bookmarking:** +- Stars help you bookmark interesting projects for later reference +- Star count indicates project popularity and community trust +- Starred repos appear in your GitHub profile, showing your interests + +**Open Source Signal:** +- Stars encourage maintainers (shows appreciation) +- High star count attracts more contributors +- Helps projects gain visibility in GitHub search and recommendations + +**Professional Context:** +- Shows you follow best practices and quality projects +- Indicates awareness of industry tools and trends + +**Why Following Matters:** + +**Networking:** +- See what other developers are working on +- Discover new projects through their activity +- Build professional connections beyond the classroom + +**Learning:** +- Learn from others' code and commits +- See how experienced developers solve problems +- Get inspiration for your own projects + +**Collaboration:** +- Stay updated on classmates' work +- Easier to find team members for future projects +- Build a supportive learning community + +**Career Growth:** +- Follow thought leaders in your technology stack +- See trending projects in real-time +- Build visibility in the developer community + +**GitHub Best Practices:** +- Star repos you find useful (not spam) +- Follow developers whose work interests you +- Engage meaningfully with the community +- Your GitHub activity shows employers your interests and involvement + +
+ +--- + +## Bonus Task — Compiled Language (2.5 pts) + +Implement the same service in a compiled language to prepare for multi-stage Docker builds (Lab 2). + +**Choose One:** +- **Go** (Recommended) - Small binaries, fast compilation +- **Rust** - Memory safety, modern features +- **Java/Spring Boot** - Enterprise standard +- **C#/ASP.NET Core** - Cross-platform .NET + +**Structure:** + +``` +app_go/ (or app_rust, app_java, etc.) +├── main.go +├── go.mod +├── README.md +└── docs/ + ├── LAB01.md # Implementation details + ├── GO.md # Language justification + └── screenshots/ +``` + +**Requirements:** +- Same two endpoints: `/` and `/health` +- Same JSON structure +- Document build process +- Compare binary size to Python + +
+💡 Go Example Skeleton + +```go +package main + +import ( + "encoding/json" + "net/http" + "os" + "runtime" + "time" +) + +type ServiceInfo struct { + Service Service `json:"service"` + System System `json:"system"` + Runtime Runtime `json:"runtime"` + Request Request `json:"request"` +} + +var startTime = time.Now() + +func mainHandler(w http.ResponseWriter, r *http.Request) { + info := ServiceInfo{ + Service: Service{ + Name: "devops-info-service", + Version: "1.0.0", + }, + System: System{ + Platform: runtime.GOOS, + Architecture: runtime.GOARCH, + CPUCount: runtime.NumCPU(), + }, + // ... implement rest + } + + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(info) +} + +func main() { + http.HandleFunc("/", mainHandler) + http.HandleFunc("/health", healthHandler) + + port := os.Getenv("PORT") + if port == "" { + port = "8080" + } + + http.ListenAndServe(":"+port, nil) +} +``` + +
+ +--- + +## How to Submit + +1. **Create Branch:** + ```bash + git checkout -b lab01 + ``` + +2. **Commit Work:** + ```bash + git add app_python/ + git commit -m "feat: implement lab01 devops info service" + git push -u origin lab01 + ``` + +3. **Create Pull Requests:** + - **PR #1:** `your-fork:lab01` → `course-repo:master` + - **PR #2:** `your-fork:lab01` → `your-fork:master` + +4. **Verify:** + - All files present + - Screenshots included + - Documentation complete + +--- + +## Acceptance Criteria + +### Main Tasks (10 points) + +**Application Functionality (3 pts):** +- [ ] Service runs without errors +- [ ] `GET /` returns all required fields: + - [ ] Service metadata (name, version, description, framework) + - [ ] System info (hostname, platform, architecture, CPU, Python version) + - [ ] Runtime info (uptime, current time, timezone) + - [ ] Request info (client IP, user agent, method, path) + - [ ] Endpoints list +- [ ] `GET /health` returns status and uptime +- [ ] Configurable via environment variables (PORT, HOST) + +**Code Quality (2 pts):** +- [ ] Clean code structure +- [ ] PEP 8 compliant +- [ ] Error handling implemented +- [ ] Logging configured + +**Documentation (3 pts):** +- [ ] `app_python/README.md` complete with all sections +- [ ] `app_python/docs/LAB01.md` includes: + - [ ] Framework justification + - [ ] Best practices documentation + - [ ] API examples + - [ ] Testing evidence + - [ ] Challenges solved + - [ ] GitHub Community section (why stars/follows matter) +- [ ] All 3 required screenshots present +- [ ] Course repository starred +- [ ] Professor and TAs followed on GitHub +- [ ] At least 3 classmates followed on GitHub + +**Configuration (2 pts):** +- [ ] `requirements.txt` with pinned versions +- [ ] `.gitignore` properly configured +- [ ] Environment variables working + +### Bonus Task (2.5 points) + +- [ ] Compiled language app implements both endpoints +- [ ] Same JSON structure as Python version +- [ ] `app_/README.md` with build/run instructions +- [ ] `app_/docs/GO.md` with language justification +- [ ] `app_/docs/LAB01.md` with implementation details +- [ ] Screenshots showing compilation and execution + +--- + +## Rubric + +| Criteria | Points | Description | +|----------|--------|-------------| +| **Functionality** | 3 pts | Both endpoints work with complete, correct data | +| **Code Quality** | 2 pts | Clean, organized, follows Python standards | +| **Documentation** | 3 pts | Complete README and lab submission docs | +| **Configuration** | 2 pts | Dependencies, environment vars, .gitignore | +| **Bonus** | 2.5 pts | Compiled language implementation | +| **Total** | 12.5 pts | 10 pts required + 2.5 pts bonus | + +**Grading Scale:** +- **10/10:** Perfect implementation, excellent documentation +- **8-9/10:** All works, good docs, minor improvements possible +- **6-7/10:** Core functionality present, basic documentation +- **<6/10:** Missing features or documentation, needs revision + +--- + +## Resources + +
+📚 Python Web Frameworks + +- [Flask Documentation](https://flask.palletsprojects.com/) +- [Flask Quickstart](https://flask.palletsprojects.com/en/3.0.x/quickstart/) +- [FastAPI Documentation](https://fastapi.tiangolo.com/) +- [FastAPI Tutorial](https://fastapi.tiangolo.com/tutorial/) +- [Django Documentation](https://docs.djangoproject.com/) + +
+ +
+🐍 Python Best Practices + +- [PEP 8 Style Guide](https://pep8.org/) +- [Python Logging Tutorial](https://docs.python.org/3/howto/logging.html) +- [Python platform module](https://docs.python.org/3/library/platform.html) +- [Python socket module](https://docs.python.org/3/library/socket.html) + +
+ +
+🔧 Compiled Languages (Bonus) + +- [Go Web Development](https://golang.org/doc/articles/wiki/) +- [Go net/http Package](https://pkg.go.dev/net/http) +- [Rust Web Frameworks](https://www.arewewebyet.org/) +- [Spring Boot Quickstart](https://spring.io/quickstart) +- [ASP.NET Core Tutorial](https://docs.microsoft.com/aspnet/core/) + +
+ +
+🛠️ Development Tools + +- [Postman](https://www.postman.com/) - API testing +- [HTTPie](https://httpie.io/) - Command-line HTTP client +- [curl](https://curl.se/) - Data transfer tool +- [jq](https://stedolan.github.io/jq/) - JSON processor + +
+ +--- + +## Looking Ahead + +This service evolves throughout the course: + +- **Lab 2:** Containerize with Docker, multi-stage builds +- **Lab 3:** Add unit tests and CI/CD pipeline +- **Lab 7-8:** Add `/metrics` endpoint for Prometheus +- **Lab 9:** Deploy to Kubernetes using `/health` probes +- **Lab 12:** Add `/visits` endpoint with file persistence +- **Lab 13:** Multi-environment deployment with GitOps + +--- + +**Good luck!** 🚀 + +> **Remember:** Keep it simple, write clean code, and document thoroughly. This foundation will carry through all 16 labs! diff --git a/labs/lab02.md b/labs/lab02.md new file mode 100644 index 0000000000..a1fac9c8a7 --- /dev/null +++ b/labs/lab02.md @@ -0,0 +1,362 @@ +# Lab 2 — Docker Containerization + +![difficulty](https://img.shields.io/badge/difficulty-beginner-success) +![topic](https://img.shields.io/badge/topic-Containerization-blue) +![points](https://img.shields.io/badge/points-10%2B2.5-orange) +![tech](https://img.shields.io/badge/tech-Docker-informational) + +> Containerize your Python app from Lab 1 using Docker best practices and publish it to Docker Hub. + +## Overview + +Take your Lab 1 application and package it into a Docker container. Learn image optimization, security basics, and the Docker workflow used in production. + +**What You'll Learn:** +- Writing production-ready Dockerfiles +- Docker best practices and security +- Image optimization techniques +- Docker Hub workflow + +--- + +## Tasks + +### Task 1 — Create Dockerfile (4 pts) + +**Objective:** Write a Dockerfile that containerizes your Python app following best practices. + +Create `app_python/Dockerfile` with these requirements: + +**Must Have:** +- Non-root user (mandatory) +- Specific base image version (e.g., `python:3.13-slim` or `python:3.12-slim`) +- Only copy necessary files +- Proper layer ordering +- `.dockerignore` file + +**Your app should work the same way in the container as it did locally.** + +
+💡 Dockerfile Concepts & Resources + +**Key Dockerfile Instructions to Research:** +- `FROM` - Choose your base image (look at python:3.13-slim, python:3.12-slim, python:3.13-alpine) +- `RUN` - Execute commands (creating users, installing packages) +- `WORKDIR` - Set working directory +- `COPY` - Copy files into the image +- `USER` - Switch to non-root user +- `EXPOSE` - Document which port your app uses +- `CMD` - Define how to start your application + +**Critical Concepts:** +- **Layer Caching**: Why does the order of COPY commands matter? +- **Non-root User**: How do you create and switch to a non-root user? +- **Base Image Selection**: What's the difference between slim, alpine, and full images? +- **Dependency Installation**: Why copy requirements.txt separately from application code? + +**Resources:** +- [Dockerfile Reference](https://docs.docker.com/engine/reference/builder/) +- [Best Practices Guide](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) +- [Python Image Variants](https://hub.docker.com/_/python) + +**Think About:** +- What happens if you copy all files before installing dependencies? +- Why shouldn't you run as root? +- How does layer caching speed up rebuilds? + +
+ +
+💡 .dockerignore Concepts + +**Purpose:** Prevent unnecessary files from being sent to Docker daemon during build (faster builds, smaller context). + +**What Should You Exclude?** +Think about what doesn't need to be in your container: +- Development artifacts (like Python's `__pycache__`, `*.pyc`) +- Version control files (`.git` directory) +- IDE configuration files +- Virtual environments (`venv/`, `.venv/`) +- Documentation that's not needed at runtime +- Test files (if not running tests in container) + +**Key Question:** Why does excluding files from the build context matter for build speed? + +**Resources:** +- [.dockerignore Documentation](https://docs.docker.com/engine/reference/builder/#dockerignore-file) +- Look at your `.gitignore` for inspiration - many patterns overlap + +**Exercise:** Start minimal and add exclusions as needed, rather than copying a huge list you don't understand. + +
+ +**Test Your Container:** + +You should be able to: +1. Build your image using the `docker build` command +2. Run a container from your image with proper port mapping +3. Access your application endpoints from the host machine + +Verify that your application works the same way in the container as it did locally. + +--- + +### Task 2 — Docker Hub (2 pts) + +**Objective:** Publish your image to Docker Hub. + +**Requirements:** +1. Create a Docker Hub account (if you don't have one) +2. Tag your image with your Docker Hub username +3. Authenticate with Docker Hub +4. Push your image to the registry +5. Verify the image is publicly accessible + +**Documentation Required:** +- Terminal output showing successful push +- Docker Hub repository URL +- Explanation of your tagging strategy + +
+💡 Docker Hub Resources + +**Useful Commands:** +- `docker tag` - Tag images for registry push +- `docker login` - Authenticate with Docker Hub +- `docker push` - Upload image to registry +- `docker pull` - Download image from registry + +**Resources:** +- [Docker Hub Quickstart](https://docs.docker.com/docker-hub/quickstart/) +- [Docker Tag Reference](https://docs.docker.com/engine/reference/commandline/tag/) +- [Best Practices for Tagging](https://docs.docker.com/develop/dev-best-practices/) + +
+ +--- + +### Task 3 — Documentation (4 pts) + +**Objective:** Document your Docker implementation with focus on understanding and decisions. + +#### 3.1 Update `app_python/README.md` + +Add a **Docker** section explaining how to use your containerized application. Include command patterns (not exact commands) for: +- Building the image locally +- Running a container +- Pulling from Docker Hub + +#### 3.2 Create `app_python/docs/LAB02.md` + +Document your implementation with these sections: + +**Required Sections:** + +1. **Docker Best Practices Applied** + - List each practice you implemented (non-root user, layer caching, .dockerignore, etc.) + - Explain WHY each matters (not just what it does) + - Include relevant Dockerfile snippets with explanations + +2. **Image Information & Decisions** + - Base image chosen and justification (why this specific version?) + - Final image size and your assessment + - Layer structure explanation + - Optimization choices you made + +3. **Build & Run Process** + - Complete terminal output from your build process + - Terminal output showing container running + - Terminal output from testing endpoints (curl/httpie) + - Docker Hub repository URL + +4. **Technical Analysis** + - Why does your Dockerfile work the way it does? + - What would happen if you changed the layer order? + - What security considerations did you implement? + - How does .dockerignore improve your build? + +5. **Challenges & Solutions** + - Issues encountered during implementation + - How you debugged and resolved them + - What you learned from the process + +--- + +## Bonus Task — Multi-Stage Build (2.5 pts) + +**Objective:** Containerize your compiled language app (from Lab 1 bonus) using multi-stage builds. + +**Why Multi-Stage?** Separate build environment from runtime → smaller final image. + +**Example Flow:** +1. **Stage 1 (Builder):** Compile the app (large image with compilers) +2. **Stage 2 (Runtime):** Copy only the binary (small image, no build tools) + +
+💡 Multi-Stage Build Concepts + +**The Problem:** Compiled language images include the entire compiler/SDK in the final image (huge!). + +**The Solution:** Use multiple `FROM` statements: +- **Stage 1 (Builder)**: Use full SDK image, compile your application +- **Stage 2 (Runtime)**: Use minimal base image, copy only the compiled binary + +**Key Concepts to Research:** +- How to name build stages (`AS builder`) +- How to copy files from previous stages (`COPY --from=builder`) +- Choosing runtime base images (alpine, distroless, scratch) +- Static vs dynamic compilation (affects what base image you can use) + +**Questions to Explore:** +- What's the size difference between your builder and final image? +- Why can't you just use the builder image as your final image? +- What security benefits come from smaller images? +- Can you use `FROM scratch`? Why or why not? + +**Resources:** +- [Multi-Stage Builds Documentation](https://docs.docker.com/build/building/multi-stage/) +- [Distroless Base Images](https://github.com/GoogleContainerTools/distroless) +- Language-specific: Search "Go static binary Docker" or "Rust alpine Docker" + +**Challenge:** Try to get your final image under 20MB. + +
+ +**Requirements:** +- Multi-stage Dockerfile in `app_go/` (or your chosen language) +- Working containerized application +- Documentation in `app_go/docs/LAB02.md` explaining: + - Your multi-stage build strategy + - Size comparison with analysis (builder vs final image) + - Why multi-stage builds matter for compiled languages + - Terminal output showing build process and image sizes + - Technical explanation of each stage's purpose + +**Bonus Points Given For:** +- Significant size reduction achieved with clear metrics +- Deep understanding of multi-stage build benefits +- Analysis of security implications (smaller attack surface) +- Explanation of trade-offs and decisions made + +--- + +## How to Submit + +1. **Create Branch:** Create a new branch called `lab02` + +2. **Commit Work:** + - Add your changes (app_python/ directory with Dockerfile, .dockerignore, updated docs) + - Commit with a descriptive message following conventional commits format + - Push to your fork + +3. **Create Pull Requests:** + - **PR #1:** `your-fork:lab02` → `course-repo:master` + - **PR #2:** `your-fork:lab02` → `your-fork:master` + +--- + +## Acceptance Criteria + +### Main Tasks (10 points) + +**Dockerfile (4 pts):** +- [ ] Dockerfile exists in `app_python/` +- [ ] Uses specific base image version +- [ ] Runs as non-root user (USER directive) +- [ ] Proper layer ordering (dependencies before code) +- [ ] Only copies necessary files +- [ ] `.dockerignore` file present +- [ ] Image builds successfully +- [ ] Container runs and app works + +**Docker Hub (2 pts):** +- [ ] Image pushed to Docker Hub +- [ ] Image is publicly accessible +- [ ] Correct tagging used +- [ ] Can pull and run from Docker Hub + +**Documentation (4 pts):** +- [ ] `app_python/README.md` has Docker section with command patterns +- [ ] `app_python/docs/LAB02.md` complete with: + - [ ] Best practices explained with WHY (not just what) + - [ ] Image information and justifications for choices + - [ ] Terminal output from build, run, and testing + - [ ] Technical analysis demonstrating understanding + - [ ] Challenges and solutions documented + - [ ] Docker Hub repository URL provided + +### Bonus Task (2.5 points) + +- [ ] Multi-stage Dockerfile for compiled language app +- [ ] Working containerized application +- [ ] Documentation in `app_/docs/LAB02.md` with: + - [ ] Multi-stage strategy explained + - [ ] Terminal output showing image sizes (builder vs final) + - [ ] Analysis of size reduction and why it matters + - [ ] Technical explanation of each stage + - [ ] Security benefits discussed + +--- + +## Rubric + +| Criteria | Points | Description | +|----------|--------|-------------| +| **Dockerfile** | 4 pts | Correct, secure, optimized | +| **Docker Hub** | 2 pts | Successfully published | +| **Documentation** | 4 pts | Complete and clear | +| **Bonus** | 2.5 pts | Multi-stage implementation | +| **Total** | 12.5 pts | 10 pts required + 2.5 pts bonus | + +**Grading:** +- **10/10:** Perfect Dockerfile, deep understanding demonstrated, excellent analysis +- **8-9/10:** Working container, good practices, solid understanding shown +- **6-7/10:** Container works, basic security, surface-level explanations +- **<6/10:** Missing requirements, runs as root, copy-paste without understanding + +--- + +## Resources + +
+📚 Docker Documentation + +- [Dockerfile Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) +- [Docker Build Reference](https://docs.docker.com/engine/reference/builder/) +- [Multi-Stage Builds](https://docs.docker.com/build/building/multi-stage/) +- [.dockerignore](https://docs.docker.com/engine/reference/builder/#dockerignore-file) + +
+ +
+🔒 Security Resources + +- [Docker Security Best Practices](https://snyk.io/blog/10-docker-image-security-best-practices/) +- [Why Non-Root Containers](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#user) +- [Distroless Images](https://github.com/GoogleContainerTools/distroless) + +
+ +
+🛠️ Tools + +- [Hadolint](https://github.com/hadolint/hadolint) - Dockerfile linter +- [Dive](https://github.com/wagoodman/dive) - Explore image layers +- [Docker Hub](https://hub.docker.com/) - Container registry + +
+ +--- + +## Looking Ahead + +- **Lab 3:** CI/CD will automatically build these Docker images +- **Lab 7-8:** Deploy containers with docker-compose for logging/monitoring +- **Lab 9:** Run these containers in Kubernetes +- **Lab 13:** ArgoCD will deploy containerized apps automatically + +--- + +**Good luck!** 🚀 + +> **Remember:** Understanding beats copy-paste. Explain your decisions, not just your actions. Run as non-root or no points! diff --git a/labs/lab03.md b/labs/lab03.md new file mode 100644 index 0000000000..0a12c43a59 --- /dev/null +++ b/labs/lab03.md @@ -0,0 +1,928 @@ +# Lab 3 — Continuous Integration (CI/CD) + +![difficulty](https://img.shields.io/badge/difficulty-beginner-success) +![topic](https://img.shields.io/badge/topic-CI/CD-blue) +![points](https://img.shields.io/badge/points-10%2B2.5-orange) +![tech](https://img.shields.io/badge/tech-GitHub%20Actions-informational) + +> Automate your Python app testing and Docker builds with GitHub Actions CI/CD pipeline. + +## Overview + +Take your containerized app from Labs 1-2 and add automated testing and deployment. Learn how CI/CD catches bugs early, ensures code quality, and automates the Docker build/push workflow. + +**What You'll Learn:** +- Writing effective unit tests +- GitHub Actions workflow syntax +- CI/CD best practices (caching, matrix builds, security scanning) +- Automated Docker image publishing +- Continuous integration for multiple applications + +**Connection to Previous Labs:** +- **Lab 1:** Test the endpoints you created +- **Lab 2:** Automate the Docker build/push workflow +- **Lab 4+:** This CI pipeline will run for all future labs + +--- + +## Tasks + +### Task 1 — Unit Testing (3 pts) + +**Objective:** Write comprehensive unit tests for your Python application to ensure reliability. + +**Requirements:** + +1. **Choose a Testing Framework** + - Research Python testing frameworks (pytest, unittest, etc.) + - Select one and justify your choice + - Install it in your `requirements.txt` or create `requirements-dev.txt` + +2. **Write Unit Tests** + - Create `app_python/tests/` directory + - Write tests for **all** your endpoints: + - `GET /` - Verify JSON structure and required fields + - `GET /health` - Verify health check response + - Test both successful responses and error cases + - Aim for meaningful test coverage (not just basic smoke tests) + +3. **Run Tests Locally** + - Verify all tests pass locally before CI setup + - Document how to run tests in your README + +
+💡 Testing Framework Guidance + +**Popular Python Testing Frameworks:** + +**pytest (Recommended):** +- Pros: Simple syntax, powerful fixtures, excellent plugin ecosystem +- Cons: Additional dependency +- Use case: Most modern Python projects + +**unittest:** +- Pros: Built into Python (no extra dependencies) +- Cons: More verbose, less modern features +- Use case: Minimal dependency projects + +**Key Testing Concepts to Research:** +- Test fixtures and setup/teardown +- Mocking external dependencies +- Testing HTTP endpoints (test client usage) +- Test coverage measurement +- Assertions and expected vs actual results + +**What Should You Test?** +- Correct HTTP status codes (200, 404, 500) +- Response data structure (JSON fields present) +- Response data types (strings, integers, etc.) +- Edge cases (invalid requests, missing data) +- Error handling (what happens when things fail?) + +**Questions to Consider:** +- How do you test a Flask/FastAPI app without starting the server? +- Should you test that `hostname` returns your actual hostname, or just that the field exists? +- How do you simulate different client IPs or user agents in tests? + +**Resources:** +- [Pytest Documentation](https://docs.pytest.org/) +- [Flask Testing](https://flask.palletsprojects.com/en/stable/testing/) +- [FastAPI Testing](https://fastapi.tiangolo.com/tutorial/testing/) +- [Python unittest](https://docs.python.org/3/library/unittest.html) + +**Anti-Patterns to Avoid:** +- Testing framework functionality instead of your code +- Tests that always pass regardless of implementation +- Tests with no assertions +- Tests that depend on external services + +
+ +**What to Document:** +- Your testing framework choice and why +- Test structure explanation +- How to run tests locally +- Terminal output showing all tests passing + +--- + +### Task 2 — GitHub Actions CI Workflow (4 pts) + +**Objective:** Create a GitHub Actions workflow that automatically tests your code and builds Docker images with proper versioning. + +**Requirements:** + +1. **Create Workflow File** + - Create `.github/workflows/python-ci.yml` in your repository + - Name your workflow descriptively + +2. **Implement Essential CI Steps** + + Your workflow must include these logical stages: + + **a) Code Quality & Testing:** + - Install dependencies + - Run a linter (pylint, flake8, black, ruff, etc.) + - Run your unit tests + + **b) Docker Build & Push with Versioning:** + - Authenticate with Docker Hub + - Build your Docker image + - Tag with proper version strategy (see versioning section below) + - Push to Docker Hub with multiple tags + +3. **Versioning Strategy** + + Choose **one** versioning approach and implement it: + + **Option A: Semantic Versioning (SemVer)** + - Version format: `v1.2.3` (major.minor.patch) + - Use git tags for releases + - Tag images like: `username/app:1.2.3`, `username/app:1.2`, `username/app:latest` + - **When to use:** Traditional software releases with breaking changes + + **Option B: Calendar Versioning (CalVer)** + - Version format: `2024.01.15` or `2024.01` (year.month.day or year.month) + - Based on release date + - Tag images like: `username/app:2024.01`, `username/app:latest` + - **When to use:** Time-based releases, continuous deployment + + **Required:** + - Document which strategy you chose and why + - Implement it in your CI workflow + - Show at least 2 tags per image (e.g., version + latest) + +4. **Workflow Triggers** + - Configure when the workflow runs (push, pull request, etc.) + - Consider which branches should trigger builds + +5. **Testing the Workflow** + - Push your workflow file and verify it runs + - Fix any issues that arise + - Ensure all steps complete successfully + - Verify Docker Hub shows your version tags + +
+💡 GitHub Actions Concepts + +**Core Concepts to Research:** + +**Workflow Anatomy:** +- `name` - What is your workflow called? +- `on` - When does it run? (push, pull_request, schedule, etc.) +- `jobs` - What work needs to be done? +- `steps` - Individual commands within a job +- `runs-on` - What OS environment? (ubuntu-latest, etc.) + +**Key Questions:** +- Should you run CI on every push, or only on pull requests? +- What happens if tests fail? Should the workflow continue? +- How do you access secrets (like Docker Hub credentials) securely? +- Why might you want multiple jobs vs multiple steps in one job? + +**Python CI Steps Pattern:** +```yaml +# This is a pattern, not exact copy-paste code +# Research the actual syntax and actions needed + +- Set up Python environment +- Install dependencies +- Run linter +- Run tests +``` + +**Docker CI Steps Pattern:** +```yaml +# This is a pattern, not exact copy-paste code +# Research the actual actions and their parameters + +- Log in to Docker Hub +- Extract metadata for tags +- Build and push Docker image +``` + +**Important Concepts:** +- **Actions Marketplace:** Reusable actions (actions/checkout, actions/setup-python, docker/build-push-action) +- **Secrets:** How to store Docker Hub credentials securely +- **Job Dependencies:** Can one job depend on another succeeding? +- **Matrix Builds:** Testing multiple Python versions (optional but good to know) +- **Caching:** Speed up workflows by caching dependencies (we'll add this in Task 3) + +**Resources:** +- [GitHub Actions Documentation](https://docs.github.com/en/actions) +- [Building and Testing Python](https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python) +- [Publishing Docker Images](https://docs.docker.com/ci-cd/github-actions/) +- [GitHub Actions Marketplace](https://github.com/marketplace?type=actions) +- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +**Security Best Practices:** +- Never hardcode passwords or tokens in workflow files +- Use GitHub Secrets for sensitive data +- Understand when secrets are exposed to pull requests from forks +- Use `secrets.GITHUB_TOKEN` for GitHub API access (auto-provided) + +**Docker Hub Authentication:** +You'll need to create a Docker Hub access token and add it as a GitHub Secret. Research: +- How to create Docker Hub access tokens +- How to add secrets to your GitHub repository +- How to reference secrets in workflow files (hint: `${{ secrets.NAME }}`) + +
+ +
+💡 Versioning Strategy Guidance + +**Semantic Versioning (SemVer):** + +**Format:** MAJOR.MINOR.PATCH (e.g., 1.2.3) +- **MAJOR:** Breaking changes (incompatible API changes) +- **MINOR:** New features (backward-compatible) +- **PATCH:** Bug fixes (backward-compatible) + +**Implementation Approaches:** +1. **Manual Git Tags:** Create git tags (v1.0.0) and reference in workflow +2. **Automated from Commits:** Parse conventional commits to bump version +3. **GitHub Releases:** Trigger on release creation + +**Docker Tagging Example:** +- `username/app:1.2.3` (full version) +- `username/app:1.2` (minor version, rolling) +- `username/app:1` (major version, rolling) +- `username/app:latest` (latest stable) + +**Pros:** Clear when breaking changes occur, industry standard for libraries +**Cons:** Requires discipline to follow rules correctly + +--- + +**Calendar Versioning (CalVer):** + +**Common Formats:** +- `YYYY.MM.DD` (e.g., 2024.01.15) - Daily releases +- `YYYY.MM.MICRO` (e.g., 2024.01.0) - Monthly with patch number +- `YYYY.0M` (e.g., 2024.01) - Monthly releases + +**Implementation Approaches:** +1. **Date-based:** Generate from current date in workflow +2. **Git SHA:** Combine with short commit SHA (2024.01-a1b2c3d) +3. **Build Number:** Use GitHub run number (2024.01.42) + +**Docker Tagging Example:** +- `username/app:2024.01` (month version) +- `username/app:2024.01.123` (with build number) +- `username/app:latest` (latest build) + +**Pros:** No ambiguity, good for continuous deployment, easier to remember +**Cons:** Doesn't indicate breaking changes + +--- + +**How to Implement in CI:** + +**Using docker/metadata-action:** +```yaml +# Pattern - research actual syntax +- name: Docker metadata + uses: docker/metadata-action + with: + # Define your tagging strategy here + # Can reference git tags, dates, commit SHAs +``` + +**Manual Tagging:** +```yaml +# Pattern - research actual syntax +- name: Generate version + run: echo "VERSION=$(date +%Y.%m.%d)" >> $GITHUB_ENV + +- name: Build and push + # Use ${{ env.VERSION }} in tags +``` + +**Questions to Consider:** +- How often will you release? (Daily? Per feature? Monthly?) +- Do users need to know about breaking changes explicitly? +- Are you building a library (use SemVer) or a service (CalVer works)? +- How will you track what's in each version? + +**Resources:** +- [Semantic Versioning](https://semver.org/) +- [Calendar Versioning](https://calver.org/) +- [Docker Metadata Action](https://github.com/docker/metadata-action) +- [Conventional Commits](https://www.conventionalcommits.org/) (for automated SemVer) + +
+ +
+💡 Debugging GitHub Actions + +**Common Issues & How to Debug:** + +**Workflow Won't Trigger:** +- Check your `on:` configuration +- Verify you pushed to the correct branch +- Look at Actions tab for filtering options + +**Steps Failing:** +- Click into the failed step to see full logs +- Check for typos in action names or parameters +- Verify secrets are configured correctly +- Test commands locally first + +**Docker Build Fails:** +- Ensure Dockerfile is in the correct location +- Check context path in build step +- Verify base image exists and is accessible +- Test Docker build locally first + +**Authentication Issues:** +- Verify secret names match exactly (case-sensitive) +- Check that Docker Hub token has write permissions +- Ensure you're using `docker/login-action` correctly + +**Debugging Techniques:** +- Add `run: echo "Debug message"` steps to understand workflow state +- Use `run: env` to see available environment variables +- Check Actions tab for detailed logs +- Enable debug logging (add `ACTIONS_RUNNER_DEBUG` secret = true) + +
+ +**What to Document:** +- Your workflow trigger strategy and reasoning +- Why you chose specific actions from the marketplace +- Your Docker tagging strategy (latest? version tags? commit SHA?) +- Link to successful workflow run in GitHub Actions tab +- Terminal output or screenshot of green checkmark + +--- + +### Task 3 — CI Best Practices & Security (3 pts) + +**Objective:** Optimize your CI workflow and add security scanning. + +**Requirements:** + +1. **Add Status Badge** + - Add a GitHub Actions status badge to your `app_python/README.md` + - The badge should show the current workflow status (passing/failing) + +2. **Implement Dependency Caching** + - Add caching for Python dependencies to speed up workflow + - Measure and document the speed improvement + +3. **Add Security Scanning with Snyk** + - Integrate Snyk vulnerability scanning into your workflow + - Configure it to check for vulnerabilities in your dependencies + - Document any vulnerabilities found and how you addressed them + +4. **Apply CI Best Practices** + - Research and implement at least 3 additional CI best practices + - Document which practices you applied and why they matter + +
+💡 CI Best Practices Guidance + +**Dependency Caching:** + +Caching speeds up workflows by reusing previously downloaded dependencies. + +**Key Concepts:** +- What should be cached? (pip packages, Docker layers, etc.) +- What's the cache key? (based on requirements.txt hash) +- When does cache become invalid? +- How much time does caching save? + +**Actions to Research:** +- `actions/cache` for general caching +- `actions/setup-python` has built-in cache support + +**Questions to Explore:** +- Where are Python packages stored that should be cached? +- How do you measure cache hit vs cache miss? +- What happens if requirements.txt changes? + +**Status Badges:** + +Show workflow status directly in your README. + +**Format Pattern:** +```markdown +![Workflow Name](https://github.com/username/repo/workflows/workflow-name/badge.svg) +``` + +Research how to: +- Get the correct badge URL for your workflow +- Make badges clickable (link to Actions tab) +- Display specific branch status + +**CI Best Practices to Consider:** + +Research and choose at least 3 to implement: + +1. **Fail Fast:** Stop workflow on first failure +2. **Matrix Builds:** Test multiple Python versions (3.12, 3.13) +3. **Job Dependencies:** Don't push Docker if tests fail +4. **Conditional Steps:** Only push on main branch +5. **Pull Request Checks:** Require passing CI before merge +6. **Workflow Concurrency:** Cancel outdated workflow runs +7. **Docker Layer Caching:** Cache Docker build layers +8. **Environment Variables:** Use env for repeated values +9. **Secrets Scanning:** Prevent committing secrets +10. **YAML Validation:** Lint your workflow files + +**Resources:** +- [GitHub Actions Best Practices](https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits) +- [Caching Dependencies](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows) +- [Security Hardening](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) + +
+ +
+💡 Snyk Integration Guidance + +**What is Snyk?** + +Snyk is a security tool that scans your dependencies for known vulnerabilities. + +**Key Concepts:** +- Vulnerability databases (CVEs) +- Severity levels (low, medium, high, critical) +- Automated dependency updates +- Security advisories + +**Integration Options:** + +1. **Snyk GitHub Action:** + - Use `snyk/actions` from GitHub Marketplace + - Requires Snyk API token (free tier available) + - Can fail builds on vulnerabilities + +2. **Snyk CLI in Workflow:** + - Install Snyk CLI in workflow + - Run `snyk test` command + - More flexible but requires setup + +**Setup Steps:** +1. Create free Snyk account +2. Get API token from Snyk dashboard +3. Add token as GitHub Secret +4. Add Snyk step to workflow +5. Configure severity threshold (what level fails the build?) + +**Questions to Explore:** +- Should every vulnerability fail your build? +- What if vulnerabilities have no fix available? +- How do you handle false positives? +- When should you break the build vs just warn? + +**Resources:** +- [Snyk GitHub Actions](https://github.com/snyk/actions) +- [Snyk Python Example](https://github.com/snyk/actions/tree/master/python-3.8) +- [Snyk Documentation](https://docs.snyk.io/integrations/ci-cd-integrations/github-actions-integration) + +**Common Issues:** +- Dependencies not installed before Snyk runs +- API token not configured correctly +- Overly strict severity settings breaking builds +- Virtual environment confusion + +**What to Document:** +- Your severity threshold decision and reasoning +- Any vulnerabilities found and your response +- Whether you fail builds on vulnerabilities or just warn + +
+ +**What to Document:** +- Status badge in README (visible proof it works) +- Caching implementation and speed improvement metrics +- CI best practices you applied with explanations +- Snyk integration results and vulnerability handling +- Terminal output showing improved workflow performance + +--- + +## Bonus Task — Multi-App CI with Path Filters + Test Coverage (2.5 pts) + +**Objective:** Set up CI for your compiled language app with intelligent path-based triggers AND add test coverage tracking. + +**Part 1: Multi-App CI (1.5 pts)** + +1. **Create Second CI Workflow** + - Create `.github/workflows/-ci.yml` for your Go/Rust/Java app + - Implement similar CI steps (lint, test, build Docker image) + - Use language-specific actions and best practices + - Apply versioning strategy (SemVer or CalVer) consistently + +2. **Implement Path-Based Triggers** + - Python workflow should only run when `app_python/` files change + - Compiled language workflow should only run when `app_/` files change + - Neither should run when only docs or other files change + +3. **Optimize for Multiple Apps** + - Ensure both workflows can run in parallel + - Consider using workflow templates (DRY principle) + - Document the benefits of path-based triggers + +**Part 2: Test Coverage Badge (1 pt)** + +4. **Add Coverage Tracking** + - Install coverage tool (`pytest-cov` for Python, coverage tool for your other language) + - Generate coverage reports in CI workflow + - Integrate with codecov.io or coveralls.io (free for public repos) + - Add coverage badge to README showing percentage + +5. **Coverage Goals** + - Document your current coverage percentage + - Identify what's not covered and why + - Set a coverage threshold in CI (e.g., fail if below 70%) + +
+💡 Path Filters & Multi-App CI + +**Why Path Filters?** + +In a monorepo with multiple apps, you don't want to run Python CI when only Go code changes. + +**Path Filter Syntax:** +```yaml +on: + push: + paths: + - 'app_python/**' + - '.github/workflows/python-ci.yml' +``` + +**Key Concepts:** +- Glob patterns for path matching +- When to include workflow file itself +- Exclude patterns (paths-ignore) +- How to test path filters + +**Questions to Explore:** +- Should changes to README.md trigger CI? +- Should changes to the root .gitignore trigger CI? +- What about changes to both apps in one commit? +- How do you test that path filters work correctly? + +**Multi-Language CI Patterns:** + +**For Go:** +- actions/setup-go +- golangci-lint for linting +- go test for testing +- Multi-stage Docker builds (from Lab 2 bonus) + +**For Rust:** +- actions-rs/toolchain +- cargo clippy for linting +- cargo test for testing +- cargo-audit for security + +**For Java:** +- actions/setup-java +- Maven or Gradle for build +- Checkstyle or SpotBugs for linting +- JUnit tests + +**Workflow Reusability:** + +Consider: +- Reusable workflows (call one workflow from another) +- Composite actions (bundle steps together) +- Workflow templates (DRY for similar workflows) + +**Resources:** +- [Path Filters](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushpull_requestpaths) +- [Reusable Workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) +- [Starter Workflows](https://github.com/actions/starter-workflows/tree/main/ci) + +
+ +
+💡 Test Coverage Tracking + +**What is Test Coverage?** + +Coverage measures what percentage of your code is executed by your tests. High coverage = more code is tested. + +**Why Coverage Matters:** +- Identifies untested code paths +- Prevents regressions (changes breaking untested code) +- Increases confidence in refactoring +- Industry standard quality metric + +**Coverage Tools by Language:** + +**Python (pytest-cov):** +```bash +# Install +pip install pytest-cov + +# Run with coverage +pytest --cov=app_python --cov-report=xml --cov-report=term + +# Generates coverage.xml for upload +``` + +**Go (built-in):** +```bash +go test -coverprofile=coverage.out ./... +go tool cover -html=coverage.out +``` + +**Rust (tarpaulin):** +```bash +cargo install cargo-tarpaulin +cargo tarpaulin --out Xml +``` + +**Java (JaCoCo with Maven/Gradle):** +```bash +mvn test jacoco:report +# or +gradle test jacocoTestReport +``` + +**Integration Services:** + +**Codecov (Recommended):** +- Free for public repos +- Beautiful visualizations +- PR comments with coverage diff +- Setup: Sign in with GitHub, add repo, upload coverage report + +**Coveralls:** +- Alternative to Codecov +- Similar features +- Different UI + +**Coverage in CI Workflow:** +```yaml +# Pattern for Python (research actual syntax) +- name: Run tests with coverage + run: pytest --cov=. --cov-report=xml + +- name: Upload to Codecov + uses: codecov/codecov-action@v3 + with: + file: ./coverage.xml +``` + +**Coverage Badge:** +```markdown +![Coverage](https://codecov.io/gh/username/repo/branch/main/graph/badge.svg) +``` + +**Setting Coverage Thresholds:** + +You can fail CI if coverage drops below a threshold: + +```yaml +# In pytest.ini or pyproject.toml +[tool:pytest] +addopts = --cov=. --cov-fail-under=70 +``` + +**Questions to Consider:** +- What's a reasonable coverage target? (70%? 80%? 90%?) +- Should you aim for 100% coverage? (Usually no - diminishing returns) +- What code is OK to leave untested? (Error handlers, config, main) +- How do you test hard-to-reach code paths? + +**Best Practices:** +- Don't chase 100% coverage blindly +- Focus on testing critical business logic +- Integration points should have high coverage +- Simple getters/setters can be skipped +- Measure coverage trends, not just absolute numbers + +**Resources:** +- [Codecov Documentation](https://docs.codecov.com/) +- [pytest-cov Documentation](https://pytest-cov.readthedocs.io/) +- [Go Coverage](https://go.dev/blog/cover) +- [Cargo Tarpaulin](https://github.com/xd009642/tarpaulin) +- [JaCoCo](https://www.jacoco.org/) + +
+ +**What to Document:** +- Second workflow implementation with language-specific best practices +- Path filter configuration and testing proof +- Benefits analysis: Why path filters matter in monorepos +- Example showing workflows running independently +- Terminal output or Actions tab showing selective triggering +- **Coverage integration:** Screenshot/link to codecov/coveralls dashboard +- **Coverage analysis:** Current percentage, what's covered/not covered, your threshold + +--- + +## How to Submit + +1. **Create Branch:** + - Create a new branch called `lab03` + - Develop your CI workflows on this branch + +2. **Commit Work:** + - Add workflow files (`.github/workflows/`) + - Add test files (`app_python/tests/`) + - Add documentation (`app_python/docs/LAB03.md`) + - Commit with descriptive message following conventional commits + +3. **Verify CI Works:** + - Push to your fork and verify workflows run + - Check that all jobs pass + - Review workflow logs for any issues + +4. **Create Pull Requests:** + - **PR #1:** `your-fork:lab03` → `course-repo:master` + - **PR #2:** `your-fork:lab03` → `your-fork:master` + - CI should run automatically on your PRs + +--- + +## Acceptance Criteria + +### Main Tasks (10 points) + +**Unit Testing (3 pts):** +- [ ] Testing framework chosen with justification +- [ ] Tests exist in `app_python/tests/` directory +- [ ] All endpoints have test coverage +- [ ] Tests pass locally (terminal output provided) +- [ ] README updated with testing instructions + +**GitHub Actions CI (4 pts):** +- [ ] Workflow file exists at `.github/workflows/python-ci.yml` +- [ ] Workflow includes: dependency installation, linting, testing +- [ ] Workflow includes: Docker Hub login, build, and push +- [ ] Versioning strategy chosen (SemVer or CalVer) and implemented +- [ ] Docker images tagged with at least 2 tags (e.g., version + latest) +- [ ] Workflow triggers configured appropriately +- [ ] All workflow steps pass successfully +- [ ] Docker Hub shows versioned images +- [ ] Link to successful workflow run provided + +**CI Best Practices (3 pts):** +- [ ] Status badge added to README and working +- [ ] Dependency caching implemented with performance metrics +- [ ] Snyk security scanning integrated +- [ ] At least 3 CI best practices applied +- [ ] Documentation complete (see Documentation Requirements section) + +### Bonus Task (2.5 points) + +**Part 1: Multi-App CI (1.5 pts)** +- [ ] Second workflow created for compiled language app (`.github/workflows/-ci.yml`) +- [ ] Language-specific linting and testing implemented +- [ ] Versioning strategy applied to second app +- [ ] Path filters configured for both workflows +- [ ] Path filters tested and proven to work (workflows run selectively) +- [ ] Both workflows can run in parallel +- [ ] Documentation explains benefits and shows selective triggering + +**Part 2: Test Coverage (1 pt)** +- [ ] Coverage tool integrated (`pytest-cov` or equivalent) +- [ ] Coverage reports generated in CI workflow +- [ ] Codecov or Coveralls integration complete +- [ ] Coverage badge added to README +- [ ] Coverage threshold set in CI (optional but recommended) +- [ ] Documentation includes coverage analysis (percentage, what's covered/not) + +--- + +## Documentation Requirements + +Create `app_python/docs/LAB03.md` with these sections: + +### 1. Overview +- Testing framework used and why you chose it +- What endpoints/functionality your tests cover +- CI workflow trigger configuration (when does it run?) +- Versioning strategy chosen (SemVer or CalVer) and rationale + +### 2. Workflow Evidence +``` +Provide links/terminal output for: +- ✅ Successful workflow run (GitHub Actions link) +- ✅ Tests passing locally (terminal output) +- ✅ Docker image on Docker Hub (link to your image) +- ✅ Status badge working in README +``` + +### 3. Best Practices Implemented +Quick list with one-sentence explanations: +- **Practice 1:** Why it helps +- **Practice 2:** Why it helps +- **Practice 3:** Why it helps +- **Caching:** Time saved (before vs after) +- **Snyk:** Any vulnerabilities found? Your action taken + +### 4. Key Decisions +Answer these briefly (2-3 sentences each): +- **Versioning Strategy:** SemVer or CalVer? Why did you choose it for your app? +- **Docker Tags:** What tags does your CI create? (e.g., latest, version number, etc.) +- **Workflow Triggers:** Why did you choose those triggers? +- **Test Coverage:** What's tested vs not tested? + +### 5. Challenges (Optional) +- Any issues you encountered and how you fixed them +- Keep it brief - bullet points are fine + +--- + +## Rubric + +| Criteria | Points | Description | +|----------|--------|-------------| +| **Unit Testing** | 3 pts | Comprehensive tests, good coverage | +| **CI Workflow** | 4 pts | Complete, functional, automated | +| **Best Practices** | 3 pts | Optimized, secure, well-documented | +| **Bonus** | 2.5 pts | Multi-app CI with path filters | +| **Total** | 12.5 pts | 10 pts required + 2.5 pts bonus | + +**Grading:** +- **10/10:** All tasks complete, CI works flawlessly, clear documentation, meaningful tests +- **8-9/10:** CI works, good test coverage, best practices applied, solid documentation +- **6-7/10:** CI functional, basic tests, some best practices, minimal documentation +- **<6/10:** CI broken or missing steps, poor tests, incomplete work + +**Quick Checklist for Full Points:** +- ✅ Tests actually test your endpoints (not just imports) +- ✅ CI workflow runs and passes +- ✅ Docker image builds and pushes successfully +- ✅ At least 3 best practices applied (caching, Snyk, status badge, etc.) +- ✅ Documentation complete but concise (no essay needed!) +- ✅ Links/evidence provided (workflow runs, Docker Hub, etc.) + +**Documentation Should Take:** 15-30 minutes to write, 5 minutes to review + +--- + +## Resources + +
+📚 GitHub Actions Documentation + +- [GitHub Actions Quickstart](https://docs.github.com/en/actions/quickstart) +- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) +- [Building and Testing Python](https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python) +- [Publishing Docker Images](https://docs.docker.com/ci-cd/github-actions/) +- [GitHub Actions Marketplace](https://github.com/marketplace?type=actions) + +
+ +
+🧪 Testing Resources + +- [Pytest Documentation](https://docs.pytest.org/) +- [Flask Testing Guide](https://flask.palletsprojects.com/en/stable/testing/) +- [FastAPI Testing Guide](https://fastapi.tiangolo.com/tutorial/testing/) +- [Python Testing Best Practices](https://realpython.com/python-testing/) + +
+ +
+🔒 Security & Quality + +- [Snyk GitHub Actions](https://github.com/snyk/actions) +- [Snyk Python Integration](https://docs.snyk.io/integrations/ci-cd-integrations/github-actions-integration) +- [GitHub Security Best Practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) +- [Dependency Scanning](https://docs.github.com/en/code-security/supply-chain-security) + +
+ +
+⚡ Performance & Optimization + +- [Caching Dependencies](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows) +- [Docker Build Cache](https://docs.docker.com/build/cache/) +- [Workflow Optimization](https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration) + +
+ +
+🛠️ CI/CD Tools + +- [act](https://github.com/nektos/act) - Run GitHub Actions locally +- [actionlint](https://github.com/rhysd/actionlint) - Lint workflow files +- [GitHub CLI](https://cli.github.com/) - Manage workflows from terminal + +
+ +--- + +## Looking Ahead + +- **Lab 4-6:** CI will validate your Terraform and Ansible code +- **Lab 7-8:** CI will run integration tests with logging/metrics +- **Lab 9-10:** CI will validate Kubernetes manifests and Helm charts +- **Lab 13:** ArgoCD will deploy what CI builds (GitOps!) +- **All Future Labs:** This pipeline is your safety net for changes + +--- + +**Good luck!** 🚀 + +> **Remember:** CI isn't about having green checkmarks—it's about catching problems before they reach production. Focus on meaningful tests and understanding why each practice matters. Think like a DevOps engineer: automate everything, fail fast, and learn from failures. diff --git a/labs/lab04.md b/labs/lab04.md new file mode 100644 index 0000000000..f7fbf676f3 --- /dev/null +++ b/labs/lab04.md @@ -0,0 +1,1508 @@ +# Lab 4 — Infrastructure as Code (Terraform & Pulumi) + +![difficulty](https://img.shields.io/badge/difficulty-beginner-success) +![topic](https://img.shields.io/badge/topic-Infrastructure%20as%20Code-blue) +![points](https://img.shields.io/badge/points-10%2B2.5-orange) +![tech](https://img.shields.io/badge/tech-Terraform%20%7C%20Pulumi-informational) + +> Provision cloud infrastructure using code with Terraform and Pulumi, comparing both approaches. + +## Overview + +Learn Infrastructure as Code (IaC) by creating virtual machines in the cloud using two popular tools: Terraform (declarative, HCL) and Pulumi (imperative, real programming languages). + +**What You'll Learn:** +- Terraform fundamentals and HCL syntax +- Pulumi fundamentals and infrastructure with code +- Cloud provider APIs and resources +- Infrastructure lifecycle management +- IaC best practices and validation +- Comparing IaC tools and approaches + +**Connection to Previous Labs:** +- **Lab 2:** Created Docker images - now we'll provision infrastructure to run them +- **Lab 3:** CI/CD for applications - now we'll add CI/CD for infrastructure +- **Lab 5:** Ansible will provision software on these VMs (you'll need a VM ready!) + +**Why Two Tools?** +By using both Terraform and Pulumi for the same task, you'll understand: +- Different IaC philosophies (declarative vs imperative) +- Tool trade-offs and use cases +- How to evaluate IaC tools for your needs + +**Important for Lab 5:** +The VM you create in this lab will be used in **Lab 5 (Ansible)** for configuration management. You have two options: +- **Option A (Recommended):** Keep your cloud VM running until you complete Lab 5 +- **Option B:** Use a local VM (see Local VM Alternative section below) + +If you choose to destroy your cloud VM after Lab 4, you can easily recreate it later using your Terraform/Pulumi code! + +--- + +## Important: Cloud Provider Selection + +### Recommended for Russia: Yandex Cloud + +Yandex Cloud offers free tier and is accessible in Russia: +- 1 VM with 20% vCPU, 1 GB RAM (free tier) +- 10 GB SSD storage +- No credit card required initially + +### Alternative Cloud Providers + +If Yandex Cloud is unavailable, choose any of these: + +**VK Cloud (Russia):** +- Russian cloud provider +- Free trial with bonus credits +- Good documentation in Russian + +**AWS (Amazon Web Services):** +- 750 hours/month free tier (t2.micro) +- Most popular globally +- Extensive documentation + +**GCP (Google Cloud Platform):** +- $300 free credits for 90 days +- Always-free tier for e2-micro +- Modern interface + +**Azure (Microsoft):** +- $200 free credits for 30 days +- Free tier for B1s instances +- Good Windows support + +**DigitalOcean:** +- Simple pricing and interface +- $200 free credits with GitHub Student Pack +- Beginner-friendly + +### Cost Management 🚨 + +**IMPORTANT - Read This:** +- ✅ **Use smallest/free tier instances only** +- ✅ **Run `terraform destroy` when done testing** +- ✅ **Consider keeping VM for Lab 5 to avoid recreation** +- ✅ **Set billing alerts if available** +- ✅ **If not using for Lab 5, delete resources after lab completion** +- ❌ **Never commit cloud credentials to Git** + +--- + +## Local VM Alternative + +If you cannot or prefer not to use cloud providers, you can use a local VM instead. This VM will need to meet specific requirements for Lab 5 (Ansible). + +### Option 1: VirtualBox/VMware VM + +**Requirements:** +- Ubuntu 24.04 LTS (recommended) or Ubuntu 22.04 LTS +- 1 GB RAM minimum (2 GB recommended) +- 10 GB disk space +- Network adapter in Bridged mode (or NAT with port forwarding) +- SSH server installed and configured +- Your SSH public key added to `~/.ssh/authorized_keys` +- Static or predictable IP address + +**Setup Steps:** +```bash +# Install SSH server (if not installed) +sudo apt update +sudo apt install openssh-server + +# Add your SSH public key +mkdir -p ~/.ssh +echo "your-public-key-here" >> ~/.ssh/authorized_keys +chmod 700 ~/.ssh +chmod 600 ~/.ssh/authorized_keys + +# Verify SSH access from your host machine +ssh username@vm-ip-address +``` + +### Option 2: Vagrant VM + +**Requirements:** +- Vagrant installed on your machine +- VirtualBox (or another Vagrant provider) + +**Basic Vagrantfile:** +```ruby +Vagrant.configure("2") do |config| + config.vm.box = "ubuntu/noble64" # Ubuntu 24.04 LTS + # Or use "ubuntu/jammy64" for Ubuntu 22.04 LTS + config.vm.network "private_network", ip: "192.168.56.10" + config.vm.provider "virtualbox" do |vb| + vb.memory = "2048" + end +end +``` + +### Option 3: WSL2 (Windows Subsystem for Linux) + +**Note:** WSL2 can work but has networking limitations. Bridged mode VM is preferred. + +**If using local VM:** +- You can skip Terraform/Pulumi cloud provider setup +- Document your local VM setup instead +- For Task 1, show VM creation (manual or Vagrant) +- For Task 2, you can skip Pulumi (or use Pulumi to manage Vagrant) +- Focus on understanding IaC concepts with cloud provider research + +**Recommended Approach:** +Even with a local VM, complete the Terraform/Pulumi tasks with a cloud provider to gain real IaC experience. You can destroy the cloud VM after Lab 4 and use your local VM for Lab 5. + +--- + +## Tasks + +### Task 1 — Terraform VM Creation (4 pts) + +**Objective:** Create a virtual machine using Terraform on your chosen cloud provider. + +**Requirements:** + +1. **Setup Terraform** + - Install Terraform CLI + - Choose and configure your cloud provider + - Set up authentication (access keys, service accounts, etc.) + - Initialize Terraform + +2. **Define Infrastructure** + + Create a `terraform/` directory with the following resources: + + **Minimum Required Resources:** + - **VM/Compute Instance** (smallest free tier size) + - **Network/VPC** (if required by provider) + - **Security Group/Firewall Rules:** + - Allow SSH (port 22) from your IP + - Allow HTTP (port 80) + - Allow custom port 5000 (for future app deployment) + - **Public IP Address** (to access VM remotely) + +3. **Configuration Best Practices** + - Use variables for configurable values (region, instance type, etc.) + - Use outputs to display important information (public IP, etc.) + - Add appropriate tags/labels for resource identification + - Use `.gitignore` for sensitive files + +4. **Apply Infrastructure** + - Run `terraform plan` to preview changes + - Review the plan carefully + - Apply infrastructure + - Verify VM is accessible via SSH + - Document the public IP and connection method + +5. **State Management** + - Keep state file local (for now) + - Understand what the state file contains + - **Never commit `terraform.tfstate` to Git** + +
+💡 Terraform Fundamentals + +**What is Terraform?** + +Terraform is a declarative IaC tool that lets you define infrastructure in configuration files (HCL - HashiCorp Configuration Language). + +**Key Concepts:** + +**Providers:** +- Plugins that interact with cloud APIs +- Each cloud has its own provider (yandex, aws, google, azurerm) +- Configure authentication and region + +**Resources:** +- Infrastructure components (VMs, networks, firewalls) +- Format: `resource "type" "name" { ... }` +- Each resource has required and optional arguments + +**Data Sources:** +- Query existing infrastructure +- Example: Find latest Ubuntu image ID +- Format: `data "type" "name" { ... }` + +**Variables:** +- Make configurations reusable +- Define in `variables.tf` +- Set values in `terraform.tfvars` (gitignored!) +- Reference: `var.variable_name` + +**Outputs:** +- Display important values after apply +- Example: VM public IP +- Define in `outputs.tf` + +**State File:** +- Tracks real infrastructure +- Maps config to reality +- **Never commit to Git** (contains sensitive data) +- Add to `.gitignore` + +**Typical Workflow:** +```bash +terraform init # Initialize provider plugins +terraform fmt # Format code +terraform validate # Check syntax +terraform plan # Preview changes +terraform apply # Create/update infrastructure +terraform destroy # Delete all infrastructure +``` + +**Resources:** +- [Terraform Documentation](https://developer.hashicorp.com/terraform/docs) +- [Terraform Registry](https://registry.terraform.io/) - Provider docs +- [HCL Syntax](https://developer.hashicorp.com/terraform/language/syntax) + +
+ +
+☁️ Yandex Cloud Terraform Guide + +**Yandex Cloud Setup:** + +**Authentication:** +- Create service account in Yandex Cloud Console +- Generate authorized key (JSON) +- Set key file path or use environment variables + +**Provider Configuration Pattern:** +```hcl +terraform { + required_providers { + yandex = { + source = "yandex-cloud/yandex" + } + } +} + +provider "yandex" { + # Configuration here (zone, folder_id, etc.) +} +``` + +**Key Resources:** +- `yandex_compute_instance` - Virtual machine +- `yandex_vpc_network` - Virtual private cloud +- `yandex_vpc_subnet` - Subnet within VPC +- `yandex_vpc_security_group` - Firewall rules + +**Free Tier Instance:** +- Platform: standard-v2 +- Cores: 2 (core_fraction = 20%) +- Memory: 1 GB +- Boot disk: 10 GB HDD + +**SSH Access:** +- Add SSH public key to `metadata` +- Use `ssh-keys` metadata field +- Connect: `ssh @` + +**Resources:** +- [Yandex Cloud Terraform Provider](https://registry.terraform.io/providers/yandex-cloud/yandex/latest/docs) +- [Getting Started Guide](https://cloud.yandex.com/en/docs/tutorials/infrastructure-management/terraform-quickstart) +- [Compute Instance Example](https://registry.terraform.io/providers/yandex-cloud/yandex/latest/docs/resources/compute_instance) + +
+ +
+☁️ AWS Terraform Guide + +**AWS Setup:** + +**Authentication:** +- Create IAM user with EC2 permissions +- Generate access key ID and secret access key +- Configure AWS CLI or use environment variables +- Never hardcode credentials + +**Provider Configuration Pattern:** +```hcl +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + } + } +} + +provider "aws" { + region = var.region # e.g., "us-east-1" +} +``` + +**Key Resources:** +- `aws_instance` - EC2 instance +- `aws_vpc` - Virtual Private Cloud +- `aws_subnet` - Subnet within VPC +- `aws_security_group` - Firewall rules +- `aws_key_pair` - SSH key + +**Free Tier Instance:** +- Instance type: t2.micro +- AMI: Amazon Linux 2 or Ubuntu (find with data source) +- 750 hours/month free for 12 months +- 30 GB storage included + +**Data Source for AMI:** +Use `aws_ami` data source to find latest Ubuntu image dynamically + +**Resources:** +- [AWS Provider Documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) +- [EC2 Instance Resource](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) +- [AWS Free Tier](https://aws.amazon.com/free/) + +
+ +
+☁️ GCP Terraform Guide + +**GCP Setup:** + +**Authentication:** +- Create service account in Google Cloud Console +- Download JSON key file +- Set `GOOGLE_APPLICATION_CREDENTIALS` environment variable +- Enable Compute Engine API + +**Provider Configuration Pattern:** +```hcl +terraform { + required_providers { + google = { + source = "hashicorp/google" + } + } +} + +provider "google" { + project = var.project_id + region = var.region +} +``` + +**Key Resources:** +- `google_compute_instance` - VM instance +- `google_compute_network` - VPC network +- `google_compute_subnetwork` - Subnet +- `google_compute_firewall` - Firewall rules + +**Free Tier Instance:** +- Machine type: e2-micro +- Zone: us-central1-a (or other free tier zone) +- Always free (within limits) +- Boot disk: 30 GB standard persistent disk + +**Resources:** +- [Google Provider Documentation](https://registry.terraform.io/providers/hashicorp/google/latest/docs) +- [Compute Instance Resource](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance) +- [GCP Free Tier](https://cloud.google.com/free) + +
+ +
+☁️ Other Cloud Providers + +**Azure:** +- Provider: `azurerm` +- Resource: `azurerm_linux_virtual_machine` +- Free tier: B1s instance +- [Azure Provider Docs](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs) + +**VK Cloud:** +- Based on OpenStack +- Provider: OpenStack provider +- [VK Cloud Documentation](https://mcs.mail.ru/help/) + +**DigitalOcean:** +- Provider: `digitalocean` +- Resource: `digitalocean_droplet` +- Simple and beginner-friendly +- [DigitalOcean Provider Docs](https://registry.terraform.io/providers/digitalocean/digitalocean/latest/docs) + +**Questions to Explore:** +- What's the smallest instance size for your provider? +- How do you find the right OS image ID? +- What authentication method does your provider use? +- How do you add SSH keys to instances? + +
+ +
+🔒 Security Best Practices + +**Credentials Management:** + +**❌ NEVER DO THIS:** +```hcl +provider "aws" { + access_key = "AKIAIOSFODNN7EXAMPLE" # NEVER! + secret_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" # NEVER! +} +``` + +**✅ DO THIS INSTEAD:** + +**Option 1: Environment Variables** +```bash +export AWS_ACCESS_KEY_ID="your-key" +export AWS_SECRET_ACCESS_KEY="your-secret" +# Provider will auto-detect +``` + +**Option 2: Credentials File** +```bash +# ~/.aws/credentials (for AWS) +[default] +aws_access_key_id = your-key +aws_secret_access_key = your-secret +``` + +**Option 3: terraform.tfvars (gitignored)** +```hcl +# terraform.tfvars (add to .gitignore!) +access_key = "your-key" +secret_key = "your-secret" +``` + +**Files to Add to .gitignore:** +``` +# Terraform +*.tfstate +*.tfstate.* +.terraform/ +terraform.tfvars +*.tfvars +.terraform.lock.hcl + +# Cloud credentials +*.pem +*.key +*.json # Service account keys +credentials +``` + +**SSH Key Management:** +- Generate SSH key pair locally +- Add public key to cloud provider +- Keep private key secure (never commit) +- Use `chmod 600` on private key file + +**Security Group Rules:** +- Restrict SSH to your IP only (not 0.0.0.0/0) +- Only open ports you need +- Document why each port is open + +
+ +
+📁 Terraform Project Structure + +**Recommended Structure:** + +``` +terraform/ +├── .gitignore # Ignore state, credentials +├── main.tf # Main resources +├── variables.tf # Input variables +├── outputs.tf # Output values +├── terraform.tfvars # Variable values (gitignored!) +└── README.md # Setup instructions +``` + +**What Goes in Each File:** + +**main.tf:** +- Provider configuration +- Resource definitions +- Data sources + +**variables.tf:** +- Variable declarations +- Descriptions +- Default values (non-sensitive only) + +**outputs.tf:** +- Important values to display +- VM IP addresses +- Connection strings + +**terraform.tfvars:** +- Actual variable values +- Secrets and credentials +- **MUST be in .gitignore** + +**Alternative: Single File** +For small projects, you can put everything in `main.tf`, but multi-file is more maintainable. + +
+ +**What to Document:** +- Cloud provider chosen and why +- Terraform version used +- Resources created (VM size, region, etc.) +- Public IP address of created VM +- SSH connection command +- Terminal output from `terraform plan` and `terraform apply` +- Proof of SSH access to VM + +--- + +### Task 2 — Pulumi VM Creation (4 pts) + +**Objective:** Destroy the Terraform VM and recreate the same infrastructure using Pulumi. + +**Requirements:** + +1. **Cleanup Terraform Infrastructure** + - Run `terraform destroy` to delete all resources + - Verify all resources are deleted in cloud console + - Document the cleanup process + +2. **Setup Pulumi** + - Install Pulumi CLI + - Choose a programming language (Python recommended, or TypeScript, Go, C#, Java) + - Initialize a new Pulumi project + - Configure cloud provider + +3. **Recreate Same Infrastructure** + + Create a `pulumi/` directory with equivalent resources: + + **Same Resources as Task 1:** + - VM/Compute Instance (same size) + - Network/VPC + - Security Group/Firewall (same rules) + - Public IP Address + + **Goal:** Functionally identical infrastructure, different tool + +4. **Apply Infrastructure** + - Run `pulumi preview` to see planned changes + - Apply infrastructure with `pulumi up` + - Verify VM is accessible via SSH + - Document the public IP + +5. **Compare Experience** + - What was easier/harder than Terraform? + - How does the code differ? + - Which approach do you prefer and why? + +
+💡 Pulumi Fundamentals + +**What is Pulumi?** + +Pulumi is an imperative IaC tool that lets you write infrastructure using real programming languages (Python, TypeScript, Go, etc.). + +**Key Differences from Terraform:** + +| Aspect | Terraform | Pulumi | +|--------|-----------|--------| +| **Language** | HCL (declarative) | Python, JS, Go, etc. (imperative) | +| **State** | Local or remote state file | Pulumi Cloud (free tier) or self-hosted | +| **Logic** | Limited (count, for_each) | Full programming language | +| **Testing** | External tools | Native unit tests | +| **Secrets** | Plain in state | Encrypted by default | + +**Key Concepts:** + +**Resources:** +- Similar to Terraform, but defined in code +- Example (Python): `vm = compute.Instance("my-vm", ...)` + +**Stacks:** +- Like Terraform workspaces +- Separate environments (dev, staging, prod) +- Each has its own config and state + +**Outputs:** +- Return values from your program +- Example: `pulumi.export("ip", vm.public_ip)` + +**Config:** +- Per-stack configuration +- Set with: `pulumi config set key value` +- Access in code: `config.get("key")` + +**Typical Workflow:** +```bash +pulumi new