diff --git a/.gitignore b/.gitignore index 09eb20a68..203c70d89 100644 --- a/.gitignore +++ b/.gitignore @@ -76,3 +76,4 @@ node_modules/ .classpath .project .settings/ +jenkins/jenkins_home/ diff --git a/README(2).md b/README(2).md new file mode 100644 index 000000000..ea69cbe99 --- /dev/null +++ b/README(2).md @@ -0,0 +1,77 @@ +# ENSF 400 - Winter 2025 - Course Project + +## Project Overview + +In this project, you will work based on a software project by incorporating/extending a complete CI/CD (Continuous Integration/Continuous Deployment) pipeline. This is based on an open-source sample application: https://github.com/7ep/demo + +This project can also be any application that requires the project of build, test, and deployment. +You will leverage GitHub for source control, Docker for containerizing your application, and a CI/CD tool (Jenkins) to automate the build, testing, and verification process. The goal is to validate every code change automatically through container builds, unit tests, code quality checks, and end-to-end functional tests. thank you + + +## Project Requirements + +By the end of this project, your group must deliver the following: + +1. Manage your project on GitHub and follow proper Git workflows (branching, pull requests, code reviews). Document the process of how you use Git workflows to collaborate with your team members. + +1. Containerize your application for builds and deployments. Upload and download your container images to a public or private image repository (e.g., Docker Hub or GitHub Container Registry). Ensure a container image is built with unique build tag(s) matching the triggering commit from any branch. + +1. Set up an automated CI/CD with Jenkins in a Codespace environment. Configure the pipeline to trigger upon pull requests merging changes into the main branch. + +1. Document the CI/CD process and provide clear instructions on replicating your environment. Submit a video demo at the end of the project. + +### Existing Pipelines + +You will also demonstrate the delivery of the following process and artifacts that come with the project. + +1. Run static analysis quality-gating using SonarQube +1. Performance testing with Jmeter +1. Security analysis with OWASP's "DependencyCheck" +1. Build Javadocs +___________________________________________________________________________________________________________________________________________________ + +# Organization of our ENSF 400 CI/CD Project + +## Team Members + +- Josral Frederick UCID: 30195360 +- Muhammad Aun Raza My UCID: 30172183 +- Natnael Tekli UCID: 30171167 +- Jaden Chow UCID: 30173676 + +## Project Description +The main objective in this project is to create software that incorporates/extends a complete CI/CD +(Continuous Integration/Continuous Deployment) pipeline. + +## Git Workflow +Our team follows a structured Git workflow to manage our project efficiently on GitHub. + +We begin by cloning the repository using git clone . Before making changes, we create a new feature branch with git checkout -b feature/feature-name to keep our work organized. + +Once changes are made, we stage and commit them using git add . and git commit -m "Description of changes", ensuring clear commit messages. + +To keep our branch up-to-date, we sync it with the main branch by switching to main, pulling the latest changes (git pull origin main), and merging them into our feature branch (git merge main). If merge conflicts arise, we resolve them before proceeding. + +After finalizing our changes, we push our branch to the remote repository using git push origin feature/feature-name. We then open a Pull Request (PR) on GitHub, providing a description of our modifications. + +Team members participate in a code review process, leaving feedback and suggesting necessary improvements. Once approved, the PR is merged into main, typically by a team lead or someone with write access. + +After merging, we delete the feature branch both locally (git branch -d feature/feature-name) and remotely (git push origin --delete feature/feature-name). To stay updated, we sync our local repository with the latest changes using git pull origin main. + +We follow GitFlow best practices, using descriptive branch names (e.g., feature/login-page, bugfix/error-handling) and ensuring every change is reviewed via a PR before merging into main. +External contributors fork the repository, create a new branch, make their changes, and open a PR to contribute to the project. + +This workflow ensures a structured and collaborative development process, keeping our codebase clean and organized. + +## Containerization + +To containerize our application, we use Docker and Docker Hub. After creating and reviewing the Dockerfile as a team, we build the Docker image with a unique tag matching the commit hash or branch name. This ensures traceability by linking each image to the exact code version it was built from. We generate the commit hash using COMMIT_HASH=$(git rev-parse --short HEAD), build the image with docker build -t natnaelt2/ensf400-demo:$COMMIT_HASH ., and push it to Docker Hub using docker push natnaelt2/ensf400-demo:$COMMIT_HASH. This process guarantees consistency and simplifies tracking across deployments. + +To deploy the application, we pull the Docker image from Docker Hub using the command docker pull natnaelt2/ensf400-demo:commit-hash. Finally, we run the Docker container using the command docker run -p 8080:8080 natnaelt2/ensf400-demo:commit-hash. Then visit the application with your browser at http://localhost:8080/demo. + +## Jenkins + +we are testing the jenkins trigger + + + diff --git a/build.gradle b/build.gradle index ff7b120e1..06d04e4c1 100644 --- a/build.gradle +++ b/build.gradle @@ -15,7 +15,7 @@ plugins { // gretty is a gradle plugin to make it easy to run a server and hotswap code at runtime. // https://plugins.gradle.org/plugin/org.gretty - id 'org.gretty' version '3.0.4' + id 'org.gretty' version '3.1.5' // provides access to a database versioning tool. id "org.flywaydb.flyway" version "6.0.8" diff --git a/docker-compose.yaml b/docker-compose.yaml new file mode 100644 index 000000000..e9c8523be --- /dev/null +++ b/docker-compose.yaml @@ -0,0 +1,50 @@ +services: + jenkins: + build: ./jenkins + privileged: true + user: root + ports: + - 8081:8080 + - 50000:50000 + container_name: jenkins + volumes: + - jenkins_home:/var/jenkins_home + - /var/run/docker.sock:/var/run/docker.sock + networks: + - dev-network + depends_on: + - sonarqube + + sonarqube: + image: sonarqube:8.9-community + container_name: sonarqube + environment: + - SONARQUBE_JDBC_URL=jdbc:postgresql://db:5432/sonar + - SONARQUBE_JDBC_USERNAME=sonar + - SONARQUBE_JDBC_PASSWORD=sonar + ports: + - "9000:9000" + volumes: + - sonarqube_data:/opt/sonarqube/data + networks: + - dev-network + + db: + image: postgres:latest + container_name: sonar-db + environment: + - POSTGRES_USER=sonar + - POSTGRES_PASSWORD=sonar + - POSTGRES_DB=sonar + volumes: + - db_data:/var/lib/postgresql/data + networks: + - dev-network + +networks: + dev-network: + +volumes: + jenkins_home: + sonarqube_data: + db_data: diff --git a/dockerfile b/dockerfile new file mode 100644 index 000000000..979f41369 --- /dev/null +++ b/dockerfile @@ -0,0 +1,6 @@ +FROM gradle:7.6.1-jdk11 +WORKDIR /ensf400-demo +COPY . . +RUN ./gradlew build +EXPOSE 8080 +CMD ["gradle", "appRun"] diff --git a/docs/BDD_video.mp4 b/docs/BDD_video.mp4 index 27c4646cf..571239ede 100644 Binary files a/docs/BDD_video.mp4 and b/docs/BDD_video.mp4 differ diff --git a/jenkins/Dockerfile b/jenkins/Dockerfile new file mode 100644 index 000000000..1a696d1df --- /dev/null +++ b/jenkins/Dockerfile @@ -0,0 +1,23 @@ +FROM jenkins/jenkins:alpine +USER root + +RUN apk add --update docker openrc + +RUN apk add --no-cache \ + openjdk11 \ + bash \ + docker \ + curl \ + unzip \ + py3-pip && \ + pip3 install --break-system-packages pipenv +ENV GRADLE_VERSION=7.6 +ENV GRADLE_HOME=/opt/gradle + +RUN mkdir -p ${GRADLE_HOME} && \ + curl -fsSL https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip -o /tmp/gradle.zip && \ + unzip /tmp/gradle.zip -d /opt/gradle && \ + rm /tmp/gradle.zip + +ENV PATH="${GRADLE_HOME}/gradle-${GRADLE_VERSION}/bin:${PATH}" +RUN gradle -v diff --git a/jenkins/Jenkinsfile b/jenkins/Jenkinsfile index d37024fe4..062ed802d 100644 --- a/jenkins/Jenkinsfile +++ b/jenkins/Jenkinsfile @@ -1,98 +1,80 @@ -// This jenkinsfile is used to run CI/CD on my local (Windows) box, no VM's needed. - pipeline { - agent any - environment { - // This is set so that the Python API tests will recognize it - // and go through the Zap proxy waiting at 9888 - HTTP_PROXY = 'http://127.0.0.1:9888' - } + environment { + HTTP_PROXY = 'http://127.0.0.1:9888' + JAVA_HOME = '/usr/lib/jvm/java-11-openjdk' + PATH = "${JAVA_HOME}/bin:${PATH}" + } stages { - // build the war file (the binary). This is the only - // place that happens. stage('Build') { + environment { + JAVA_HOME = '/usr/lib/jvm/java-11-openjdk' + PATH = "${JAVA_HOME}/bin:${PATH}" + } steps { sh './gradlew clean assemble' } } - // run all the unit tests - these do not require anything else - // to be running and most run very quickly. stage('Unit Tests') { + environment { + JAVA_HOME = '/usr/lib/jvm/java-11-openjdk' + PATH = "${JAVA_HOME}/bin:${PATH}" + } steps { sh './gradlew test' } - post { - always { - junit 'build/test-results/test/*.xml' - } - } } - // run the tests which require connection to a - // running database. stage('Database Tests') { + environment { + JAVA_HOME = '/usr/lib/jvm/java-11-openjdk' + PATH = "${JAVA_HOME}/bin:${PATH}" + } steps { sh './gradlew integrate' } - post { - always { - junit 'build/test-results/integrate/*.xml' - } - } } - // These are the Behavior Driven Development (BDD) tests - // See the files in src/bdd_test - // These tests do not require a running system. stage('BDD Tests') { + environment { + JAVA_HOME = '/usr/lib/jvm/java-11-openjdk' + PATH = "${JAVA_HOME}/bin:${PATH}" + } steps { sh './gradlew generateCucumberReports' - // generate the code coverage report for jacoco - sh './gradlew jacocoTestReport' } - post { - always { - junit 'build/test-results/bdd/*.xml' - } - } } - // Runs an analysis of the code, looking for any - // patterns that suggest potential bugs. stage('Static Analysis') { + environment { + JAVA_HOME = '/usr/lib/jvm/java-11-openjdk' + PATH = "${JAVA_HOME}/bin:${PATH}" + } steps { - sh './gradlew sonarqube' - // wait for sonarqube to finish its analysis - sleep 5 - sh './gradlew checkQualityGate' + withCredentials([string(credentialsId: 'sonar-token', variable: 'SONAR_TOKEN')]) { + sh ''' + ./gradlew sonarqube \ + -Dsonar.host.url=http://sonarqube:9000 \ + -Dsonar.login=admin \ + -Dsonar.password=manutd + ''' + } } } - - // Move the binary over to the test environment and - // get it running, in preparation for tests that - // require a whole system to be running. stage('Deploy to Test') { steps { - sh './gradlew deployToTestWindowsLocal' - // pipenv needs to be installed and on the path for this to work. - sh 'PIPENV_IGNORE_VIRTUALENVS=1 pipenv install' - - // Wait here until the server tells us it's up and listening - sh './gradlew waitForHeartBeat' - - // clear Zap's memory for the incoming tests - sh 'curl http://zap/JSON/core/action/newSession -s --proxy localhost:9888' + sh './gradlew deployToTestWindowsLocal' + sh 'PIPENV_IGNORE_VIRTUALENVS=1 pipenv install' + sh './gradlew waitForHeartBeat' + sh 'curl http://zap/JSON/core/action/newSession -s --proxy localhost:9888' } } - - // Run the tests which investigate the functioning of the API. stage('API Tests') { steps { sh './gradlew runApiTests' @@ -104,12 +86,6 @@ pipeline { } } - // We use a BDD framework for some UI tests, Behave, because Python rules - // when it comes to experimentation with UI tests. You can try things and see how they work out. - // this set of BDD tests does require a running system. - // BDD at the UI level is just to ensure that basic capabilities work, - // not that every little detail of UI functionality is correct. For - // that purpose, see the following stage, "UI Tests" stage('UI BDD Tests') { steps { sh './gradlew runBehaveTests' @@ -122,68 +98,38 @@ pipeline { } } - // This set of tests investigates the functionality of the UI. - // Note that this is separate fom the UI BDD Tests, which - // only focuses on essential capability and therefore only - // covers a small subset of the possibilities of UI behavior. stage('UI Tests') { - steps { - sh 'cd src/ui_tests/java && ./gradlew clean test' - } - post { - always { - junit 'src/ui_tests/java/build/test-results/test/*.xml' - } + steps { + sh 'cd src/ui_tests/java && ./gradlew clean test' + } + post { + always { + junit 'src/ui_tests/java/build/test-results/test/*.xml' } + } } - // Run OWASP's "DependencyCheck". https://owasp.org/www-project-dependency-check/ - // You are what you eat - and so it is with software. This - // software consists of a number of software by other authors. - // For example, for this project we use language tools by Apache, - // password complexity analysis, and several others. Each one of - // these might have security bugs - and if they have a security - // bug, so do we! - // - // DependencyCheck looks at the list of known - // security vulnerabilities from the United States National Institute of - // Standards and Technology (NIST), and checks if the software - // we are importing has any major known vulnerabilities. If so, - // the build will halt at this point. stage('Security: Dependency Analysis') { steps { - sh './gradlew dependencyCheckAnalyze' + sh './gradlew dependencyCheckAnalyze' } } - // Run Jmeter performance testing https://jmeter.apache.org/ - // This test simulates 50 users concurrently using our software - // for a set of common tasks. stage('Performance Tests') { steps { - sh './gradlew runPerfTests' + sh './gradlew runPerfTests' } } - // Runs mutation testing against some subset of our software - // as a spot test. Mutation testing is where bugs are seeded - // into the software and the tests are run, and we see which - // tests fail and which pass, as a result. - // - // what *should* happen is that where code or tests are altered, - // the test should fail, shouldn't it? However, it sometimes - // happens that no matter how code is changed, the tests - // continue to pass, which implies that the test wasn't really - // providing any value for those lines. stage('Mutation Tests') { steps { - sh './gradlew pitest' + sh './gradlew pitest' } } stage('Build Documentation') { steps { - sh './gradlew javadoc' + sh './gradlew javadoc' } } @@ -194,18 +140,11 @@ pipeline { } } - - // This is the stage where we deploy to production. If any test - // fails, we won't get here. Note that we aren't really doing anything - this - // is a token step, to indicate whether we would have deployed or not. Nothing actually - // happens, since this is a demo project. stage('Deploy to Prod') { steps { - // just a token operation while we pretend to deploy sh 'sleep 5' } } } - }