feat: [#14] Implement infrastructure/application deployment separation + integration testing fixes #18
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Update core principles to focus on local/production parity - Remove staging environment from environment standardization tasks - Simplify directory structure to exclude staging-specific files - Update testing strategy to focus on two-environment approach - Maintain scope manageable while achieving twelve-factor compliance
…ory deployment ## 🎯 Integration Testing Workflow Complete ### ✅ Core Improvements - **Local repository deployment**: deploy-app.sh now uses git archive instead of GitHub clone - **SSH authentication**: Fixed cloud-init and deployment scripts for reliable key-based auth - **Endpoint validation**: Corrected health checks for nginx proxy architecture - **Database migration**: Successfully migrated local environment from SQLite to MySQL - **Health validation**: All 14 health checks now pass (100% success rate) ### 🛠️ New Scripts Created - infrastructure/scripts/provision-infrastructure.sh - VM infrastructure provisioning - infrastructure/scripts/deploy-app.sh - Application deployment with local repo support - infrastructure/scripts/health-check.sh - Comprehensive endpoint and service validation ### 📋 Workflow Commands - make infra-apply ENVIRONMENT=local # Deploy VM infrastructure - make app-deploy ENVIRONMENT=local # Deploy application from local changes - make health-check ENVIRONMENT=local # Validate deployment (14/14 tests) - make infra-destroy ENVIRONMENT=local # Clean up infrastructure ### 📚 Documentation Reorganization - Moved twelve-factor status files to infrastructure/docs/refactoring/twelve-factor-refactor/ - Clarified that twelve-factor configuration management is still pending - Updated integration testing guide for new workflow - Created accurate status documentation ### 🔧 Technical Fixes - Fixed SSH BatchMode and key configuration in cloud-init - Corrected nginx proxy endpoint validation (health_check, API stats, tracker) - Updated Grafana port mapping (3000 → 3100) - Implemented MySQL connectivity validation - Enhanced error handling and logging throughout scripts ## 🚧 Twelve-Factor Status Integration testing workflow is operational. Core twelve-factor configuration management (environment templates, config processing) is next milestone. Closes partial work on #14 - integration testing workflow improvements
- Update current-status.md to reflect true state (IN PROGRESS, not COMPLETED) - Update integration-testing-improvements.md to focus on recent workflow fixes - Fix deploy-app.sh endpoint validation for nginx proxy paths and MySQL
- Consolidate all twelve-factor refactoring docs into single comprehensive README - Include current status, implementation plan, migration guide, and technical details - Remove redundant individual files (current-status.md, integration-testing-improvements.md, migration-guide.md, phase-1-implementation.md) - Update navigation documentation to reflect consolidated structure - All content now in infrastructure/docs/refactoring/twelve-factor-refactor/README.md
josecelano
force-pushed
the
14-phase-2-12-factor-app-refactoring-part-2
branch
from
e75c138 to
a9a5bcb
Compare
- Fix twelve-factor interpretation in contributor guide (infrastructure ≠ Build stage) - Clarify separation between infrastructure provisioning and app deployment - Update repository structure documentation to match actual project layout - Fix spelling/corruption errors in copilot instructions - Update Makefile command comments for correct twelve-factor terminology - Update integration testing guide to clarify workflow stages - Consolidate refactoring documentation with twelve-factor clarifications
- Fix Statistics API authentication to use query parameter (?token=) instead of Bearer token - Update nginx proxy test to use /health_check endpoint instead of generic / - Make smoke testing mandatory for E2E test success (fail if any smoke test fails) - Add comprehensive failure reporting with error counts and debugging info - Update smoke testing guide documentation with authentication examples - Update test strategy documentation to reflect mandatory smoke testing The E2E test now validates all critical tracker functionality: - Health Check API (nginx proxy port 80) - Statistics API with proper authentication (nginx proxy port 80) - UDP tracker connectivity (ports 6868, 6969) - HTTP tracker via nginx proxy (/health_check endpoint) - Direct tracker health check (port 1212) All smoke tests must pass for deployment to be considered successful.
…ests - Add test-ci and test-local targets to Makefile for clear test separation - Update GitHub Actions workflow to run make test-ci with all dependencies - Create orchestration scripts for CI (test-ci.sh) and local (test-local.sh) testing - Add unit test scripts for config, scripts, and infrastructure validation - Remove deprecated test-integration.sh and test-local-setup.sh - Document testing strategy in ci-vs-local-test-analysis.md - Update infrastructure test documentation and project references - Improve Makefile help output to clarify testing workflow This enables running syntax validation, config validation, and unit tests in GitHub Actions while keeping full E2E infrastructure tests for local development with virtualization support.
…g guide - Add 'Automated Testing Alternative' section after Overview - Add 'Automated Testing' tip after completion message - Reference tests/test-e2e.sh script with usage examples - Explain benefits and when to use automated vs manual testing - Include environment variables for customizing automated tests - Preserve all existing manual testing documentation and procedures
- Add automatic OpenTofu/Terraform initialization in test-unit-config.sh - Fixes CI workflow failure where 'tofu validate' requires 'tofu init' first - Check for .terraform directory existence before running validation - Initialize silently to avoid test output clutter - Maintains backward compatibility with already-initialized environments - Resolves GitHub Actions workflow validation errors This ensures the configuration validation test works correctly in both: - Local environments (where tofu init has been run manually) - CI environments (where the working directory is clean) Tested scenarios: - Pre-initialized environment (existing behavior preserved) - Clean environment (auto-initialization works correctly) - Full CI test suite passes with this fix
josecelano
force-pushed
the
14-phase-2-12-factor-app-refactoring-part-2
branch
from
58292ed to
f6f7a93
Compare
- Move Docker Compose tests to application/tests/ layer - Move Makefile and project-wide tests to tests/ layer - Keep infrastructure tests in infrastructure/tests/ layer - Create comprehensive test organization documentation - Add layer-specific test scripts and README files - Establish clear separation of concerns for test responsibilities This fixes the mixed-layer test organization where infrastructure/tests/ contained tests belonging to different architectural layers, violating the separation of concerns principle. New test structure: - infrastructure/tests/ - Infrastructure provisioning validation - application/tests/ - Application deployment validation - tests/ - Project-wide and cross-cutting validation Includes governance documentation to prevent future misorganization.
Replace linting-only requirement with comprehensive CI test suite that includes: - Linting validation (YAML, shell scripts, markdown) - Infrastructure tests (Terraform/OpenTofu syntax, cloud-init templates) - Application tests (Docker Compose syntax, app configuration) - Project tests (Makefile syntax, project structure, tool requirements) This ensures more comprehensive validation before commits while excluding only the slower E2E tests (~5-8 minutes) which are still recommended before pushing changes. Benefits: - Earlier detection of issues across all test layers - Better code quality through comprehensive pre-commit validation - Faster CI/CD feedback by catching issues locally - Consistent validation standards for all contributors
Refactors the infrastructure script unit tests for improved scalability and maintainability.
- Splits the monolithic test file into individual test files for each script in 'infrastructure/scripts'.
- Creates a shared 'test-utils.sh' to reduce code duplication.
- Moves all script-related test files into a new 'infrastructure/tests/scripts' subfolder.
- Updates the main test orchestrator to delegate to the new individual test files.
- Adjusts the linting script ('scripts/lint.sh') to correctly handle the new test structure and avoid false positives.
All tests, including the full end-to-end suite, pass successfully after these changes.
- Refactored the wait_for_vm_ready function in the E2E test into two more specific functions: wait_for_cloud_init_to_finish and wait_for_app_deployment_to_finish. - Improved the application health check logic to be more robust by parsing 'docker compose ps' output directly, avoiding issues with '--filter' on different Docker Compose versions. - This makes the E2E tests more reliable and easier to debug.
…h checks - Added comprehensive wait_for_services logic to check health status for all containers - Improved logging with color-coded warnings and debug output for container status - Added wait_for_system_ready to ensure cloud-init and Docker are ready before deployment - Updated deployment logic to preserve storage folder across deployments - Fixed SSH command usage with -n flag for reliability - Refactored health check detection using docker inspect for accurate status - Removed duplicate health check logic from E2E test script - Enhanced container startup validation to wait for all services to be healthy - Increased health check timeout for better reliability with fresh deployments This resolves issues where deployment script would declare success too early, only checking one container instead of waiting for all containers to be healthy. The improvements ensure MySQL and tracker containers are fully ready before running health checks and E2E tests.
The frontend_network was inherited from upstream repo that included frontend services, but this demo only deploys backend services (tracker, proxy, database, monitoring) that all communicate through the backend_network. - Remove frontend_network from proxy service networks - Remove frontend_network definition from networks section - Simplifies architecture while maintaining all functionality
Add status badge for the testing.yml workflow to provide visibility into the current CI/CD pipeline status at the top of the README file.
- Create shared shell utilities file (scripts/shell-utils.sh) with: - Centralized color variables and logging functions - Tee logging support via SHELL_UTILS_LOG_FILE - Debug and trace logging levels - Additional utility functions for common tasks - Refactor 12 shell scripts to use shared utilities: - Remove ~200 lines of duplicate color/logging code - Standardize logging patterns across all scripts - Maintain backward compatibility and test coverage - Add comprehensive documentation: - Migration summary with patterns and benefits - Usage examples and future recommendations - Validation: - All syntax validation passes (ShellCheck, yamllint, markdownlint) - All CI tests pass (make test-ci) - Full E2E tests pass (make test) - Net code reduction: ~150 lines
…ions - Add sudo cache management functions to scripts/shell-utils.sh - is_sudo_cached(): Check if sudo credentials are cached - ensure_sudo_cached(): Warn user and cache sudo credentials upfront - run_with_sudo(): Run commands with pre-cached sudo - clear_sudo_cache(): Clear sudo cache for testing - Update infrastructure scripts to use proactive sudo caching - infrastructure/scripts/fix-volume-permissions.sh: Cache sudo before operations - infrastructure/scripts/provision-infrastructure.sh: Cache sudo before tofu apply - tests/test-e2e.sh: Prepare sudo cache before infrastructure provisioning - Improve user experience for 'make test' command - Password prompt now appears clearly at the beginning - No more mixed output with OpenTofu verbose logs - Clear messaging about when and why sudo is needed - Leverages standard sudo timeout (~15 minutes) - Add comprehensive documentation - ADR-005: Sudo Cache Management for Infrastructure Operations - Documents chosen approach and 7 alternatives considered - Updated .github/copilot-instructions.md with implementation details - Updated docs/README.md with new ADR reference - Update Makefile with improved user guidance for sudo operations Resolves password prompt mixing issue during infrastructure testing while maintaining security through standard sudo timeout mechanism.
15 tasks
15 tasks
…guration - Create dedicated ADR documentation (docs/adr/README.md) - Add ADR guidelines, template, and lessons learned - Move ADR list from docs/README.md to dedicated location - Document best practices for keeping ADRs focused - Configure global table line length exclusion in markdownlint - Update .markdownlint.json to exclude tables from MD013 rule - Create .markdownlint.md with configuration documentation - Update .github/copilot-instructions.md with simplified table guidance - Remove unnecessary markdownlint ignore blocks from existing tables - Document SSH host key verification troubleshooting - Create docs/infrastructure/ssh-host-key-verification.md - Provide comprehensive solution for VM development warnings - Improve documentation structure and navigation - Update docs/README.md with cleaner organization - Add cross-references and proper categorization - Include reference to markdownlint configuration guidelines Benefits: - Tables automatically ignore line length limits (no manual ignore blocks needed) - Cleaner markdown files without visual clutter - Better organized ADR documentation with clear guidelines - Comprehensive troubleshooting documentation for common issues
- Update provision-infrastructure test to use invalid environment parameter - Prevents script from reaching sudo caching logic during CI testing - Test now fails early during parameter validation instead of at infrastructure stage - Maintains error handling test coverage without requiring interactive sudo prompts Problem: - make test-ci was prompting for sudo password when cache expired - Caused by test calling provision-infrastructure.sh with parameters that trigger apply action - Apply action requires sudo for libvirt operations via ensure_sudo_cached() Solution: - Changed test to use 'invalid-env' parameter instead of 'local' - Script fails during environment validation before reaching sudo logic - CI tests now run completely non-interactively Benefits: - CI tests run without user interaction - Faster test execution (3s vs 19s) - Maintains test validation of error handling behavior - Clean separation between CI tests and system operations
- Add ssh-utils.sh script for managing SSH host key verification issues - Integrate SSH cleanup into infrastructure provisioning workflow - Add Makefile targets for SSH troubleshooting (ssh-clean, ssh-prepare) - Update documentation with SSH troubleshooting guidance SSH Utilities (infrastructure/scripts/ssh-utils.sh): - clean_vm_known_hosts() - Remove host keys for specific VM IP - clean_libvirt_known_hosts() - Clean entire libvirt network range - prepare_ssh_connection() - Comprehensive SSH preparation workflow - Support for both specific IP and network-wide cleanup Infrastructure Integration: - Auto-clean SSH known_hosts before and after VM provisioning - Prevent host key verification warnings during deployment - Non-critical operations (won't fail deployment if SSH cleanup fails) Makefile Enhancements: - make ssh-clean: Fix host key verification warnings - make ssh-prepare: Clean and test SSH connectivity - Updated help documentation and troubleshooting guide Benefits: - Eliminates common SSH host key verification warnings - Smoother VM development workflow - Better developer experience with local testing - Automated SSH maintenance during infrastructure operations
|
ACK aa968d0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat: [#14] Comprehensive twelve-factor deployment system with advanced developer experience
This PR implements a complete twelve-factor application deployment system with sophisticated developer experience improvements, comprehensive testing infrastructure, and advanced automation capabilities.
🎯 Main Achievements
1. Twelve-Factor Application Deployment (Core Implementation)
Complete separation of infrastructure provisioning and application deployment:
2. Advanced Developer Experience System
SSH Host Key Management Automation:
infrastructure/scripts/ssh-utils.sh)make ssh-clean,make ssh-prepareIntelligent Sudo Cache Management:
3. Centralized Shell Utilities & Code Quality
Major Refactoring Achievement:
scripts/shell-utils.sh)🏗️ Architecture Enhancements
New Infrastructure Scripts
infrastructure/scripts/provision-infrastructure.sh- Pure infrastructure provisioninginfrastructure/scripts/deploy-app.sh- Application deployment with local repository supportinfrastructure/scripts/health-check.sh- 14-point comprehensive validationinfrastructure/scripts/ssh-utils.sh- SSH troubleshooting automationThree-Layer Testing Architecture
Enhanced Container Orchestration
🔧 Integration Testing Fixes
1. Local Repository Deployment
deploy-app.shnow usesgit archiveinstead of GitHub clone2. SSH Authentication & Connectivity
BatchMode=yesfor reliable automation3. Endpoint Validation & Database
📊 Validation Results
🧪 Testing Infrastructure Improvements
CI/CD Integration
Test Organization & Coverage
📚 Documentation & Architecture Decisions
New Architecture Decision Records (ADRs)
docs/adr/README.md)Comprehensive Documentation
🔄 Backward Compatibility & Migration
Legacy commands maintained with helpful guidance:
Smooth migration path for existing workflows while encouraging twelve-factor adoption.
📈 Impact & Metrics
🎯 Future Foundation
This implementation establishes the complete foundation for the twelve-factor configuration management system. The infrastructure/application separation is operational and ready for:
✅ Quality Assurance