From df1822e84c2d26ade826379361716699daf94f37 Mon Sep 17 00:00:00 2001 From: Francisco Guerrero Date: Sat, 23 Aug 2025 16:00:41 -0700 Subject: [PATCH 1/6] CASSSIDECAR-339: Add developer-facing docs on testing Sidecar Patch by Francisco Guerrero; reviewed by TBD for CASSSIDECAR-339 --- CONTRIBUTING.md | 12 ++ TESTING.md | 358 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 370 insertions(+) create mode 100644 TESTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 85829de28..acd18b3ff 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -28,6 +28,7 @@ We warmly welcome and appreciate contributions from the community. * [Discuss](#discuss) * [Create a Ticket](#ticket) * [Integration with IntelliJ IDEA](#idea) + * [Testing](#testing) * [Source Code Best Practices](#best-practices) * [Introducing new APIs](#new-apis) * [Asynchronous Programming](#async-programming) @@ -76,6 +77,17 @@ You can then use the provided configuration that adheres to the project source f > **NOTE**: Opening a newly cloned repository in IDEA before running the command above will result in the default code > format settings being used instead; if that is the case, delete the `.idea` directory and start over. +### Testing + +For comprehensive information about running tests, including unit tests, integration tests, and test configuration, +please refer to the [Testing Guide](TESTING.md). This guide covers: + +- Test prerequisites and setup +- Different types of tests (unit, integration, container) +- Running tests with various configurations +- Troubleshooting common testing issues +- Development testing best practices + ## Source Code Best Practices The Apache Cassandra Sidecar project uses the [vertx](https://vertx.io) toolkit. It uses the asynchronous and diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 000000000..2af9042bd --- /dev/null +++ b/TESTING.md @@ -0,0 +1,358 @@ + + +# Testing Guide for Apache Cassandra Sidecar + +This document provides a comprehensive guide to running tests for the Apache Cassandra Sidecar project. + +## Table of Contents + +- [Prerequisites](#prerequisites) +- [Test Types](#test-types) +- [Running Tests](#running-tests) +- [Test Configuration](#test-configuration) +- [Development Testing](#development-testing) +- [Continuous Integration](#continuous-integration) +- [Troubleshooting](#troubleshooting) + +## Prerequisites + +### Required Software + +1. **Java 11 or higher** (OpenJDK or Oracle) +2. **Docker** for running integration tests +3. **Git** for cloning repositories + +### Build Prerequisites + +Before running tests, you must build the required Cassandra dtest jars: + +```bash +./scripts/build-dtest-jars.sh +``` + +This script builds dtest jars for the supported Cassandra versions. +You can customize which versions to build: + +- **REPO**: Cassandra git repository (default: `https://github.com/apache/cassandra.git`) +- **BRANCHES**: Space-delimited list of branches (default: `"cassandra-4.0 cassandra-4.1 cassandra-5.0 trunk"`) + +Example with custom branches: +```bash +BRANCHES="cassandra-4.1 trunk" ./scripts/build-dtest-jars.sh +``` + +### Network Setup for Multi-Node Tests + +For multi-node in-jvm dtests, network aliases must be configured. +Tests assume each node's IP address is `127.0.0.x`, where `x` is the node ID. + +#### macOS Network Aliases + +Create temporary aliases for nodes 2-20: +```bash +for i in {2..20}; do sudo ifconfig lo0 alias "127.0.0.${i}"; done +``` + +#### Linux Network Aliases + +Create temporary aliases for nodes 2-20: +```bash +for i in {2..20}; do sudo ip addr add "127.0.0.${i}/8" dev lo; done +``` + +## Test Types + +### Unit Tests + +Unit tests verify individual components in isolation. They are fast-running +and don't require external dependencies. + +**Location**: `src/test/java/` in each module + +**Running unit tests**: +```bash +# Run all unit tests +./gradlew test + +# Run unit tests for a specific module +./gradlew :client:test +./gradlew :server:test +``` + +### Integration Tests + +Integration tests verify component interactions and require running +Cassandra instances. They are divided into two categories: + +#### Lightweight Integration Tests +- Basic functionality tests +- Faster execution +- **Tag**: None (default) + +#### Heavyweight Integration Tests +- Complex scenarios +- Multi-node cluster tests +- Longer execution times +- **Tag**: `@Tag("heavy")` + +**Location**: `src/integrationTest/` in test modules + +**Running integration tests**: +```bash +# Run all integration tests +./gradlew integrationTest + +# Run only lightweight integration tests +./gradlew integrationTestLightWeight + +# Run only heavyweight integration tests +./gradlew integrationTestHeavyWeight + +# Run integration tests for specific module +./gradlew :integration-tests:integrationTest + +# Run integration tests for specific module +./gradlew :server:integrationTest +``` + +### Container Tests + +Container-based tests run in Docker environments for additional isolation. + +**Location**: `src/containerTest/` in relevant modules + +### Test Fixtures + +Test fixtures provide shared test utilities and data across modules. + +**Location**: `src/testFixtures/` in modules with `java-test-fixtures` plugin + +## Running Tests + +### Quick Test Commands + +```bash +# Run all tests (unit + integration) +./gradlew check + +# Run only unit tests +./gradlew test + +# Run only integration tests +./gradlew integrationTest + +# Skip integration tests +./gradlew check -x integrationTest + +# Run with specific Cassandra versions +./gradlew test -Dcassandra.sidecar.versions_to_test=4.1,5.1 +``` + +### Test Execution Configuration + +Integration tests support several environment variables: + +| Variable | Default | Description | +|----------|---------|-------------| +| `INTEGRATION_MAX_HEAP_SIZE` | `3000M` | Maximum heap size for integration tests | +| `INTEGRATION_MAX_PARALLEL_FORKS` | `4` | Number of parallel test forks | +| `INTEGRATION_MTLS_ENABLED` | `true` | Enable mTLS for integration tests | + +Example: +```bash +INTEGRATION_MAX_HEAP_SIZE=4000M INTEGRATION_MAX_PARALLEL_FORKS=2 ./gradlew integrationTest +``` + +### Gradle Properties + +You can also skip integration tests using Gradle properties: + +```bash +# Via command line +./gradlew check -x integrationTest + +# Via environment variable +export skipIntegrationTest=true +./gradlew check +``` + +## Test Configuration + +### Cassandra Version Testing + +The test framework supports multiple Cassandra versions simultaneously. Configure versions via: + +**System Property**: +```bash +-Dcassandra.sidecar.versions_to_test=4.0,4.1,5.1 +``` + +**Default versions**: `4.1,5.1` (as defined in `TestVersionSupplier.java`) + +### Test Logging + +Integration tests use a dedicated logback configuration: +- **File**: `server/src/test/resources/logback-in-jvm-dtest.xml` +- **Level**: Configurable per test class + +### JVM Options + +Tests running on Java 11+ automatically receive optimized JVM arguments defined in `gradle/common/java11Options.gradle`. + +## Development Testing + +### Running Tests During Development + +For faster development cycles: + +```bash +# Run with code checks disabled +./gradlew run + +# Run specific test class +./gradlew test --tests "org.apache.cassandra.sidecar.HealthServiceTest" + +# Run specific integration test class +./gradlew :integration-tests:integrationTestLightWeight --tests "org.apache.cassandra.sidecar.routes.RoutesIntegrationTest" + +# Or run a heavy-weight test class +./gradlew :server:integrationTestHeavyWeight --tests "org.apache.cassandra.sidecar.routes.tokenrange.LeavingTest" + +# Debug test execution +./gradlew test --debug-jvm + +# Run tests continuously +./gradlew test --continuous +``` + +### Test Reports + +Test reports are generated in: +- **Unit tests**: `build/test-results/` +- **Integration tests**: `build/test-results/integration/` +- **HTML reports**: Available in respective `reports/` directories + +### Code Coverage + +Jacoco coverage reports are automatically generated after tests: + +```bash +# Generate coverage report +./gradlew jacocoTestReport + +# View coverage report +open build/reports/jacoco/test/html/index.html +``` + +## Continuous Integration + +### CircleCI Integration + +The project uses CircleCI for automated testing. To set up CircleCI on your fork: + +1. Use CircleCI's "Add Projects" function +2. Choose manual configuration (don't replace the existing config) +3. CircleCI will automatically use the in-project configuration + +### Code Quality Checks + +Before tests run, the following quality checks execute: +- **Checkstyle**: Code style verification +- **SpotBugs**: Static analysis for bugs +- **RAT**: License header verification + +```bash +# Run code quality checks only +./gradlew codeCheckTasks + +# Skip code quality checks +./gradlew test -x codeCheckTasks +``` + +## Troubleshooting + +### Common Issues + +#### "DTest jar not found" +**Solution**: Run `./scripts/build-dtest-jars.sh` to build required dependencies. + +#### "Network interface not available" +**Solution**: Set up network aliases as described in [Network Setup](#network-setup-for-multi-node-tests). + +#### "OutOfMemoryError during integration tests" +**Solution**: Increase heap size: +```bash +INTEGRATION_MAX_HEAP_SIZE=4000M ./gradlew integrationTest +``` + +#### "Tests are hanging or slow" +**Solution**: Reduce parallel forks: +```bash +INTEGRATION_MAX_PARALLEL_FORKS=1 ./gradlew integrationTest +``` + +### Test Isolation Issues + +Integration tests use `forkEvery = 1` to ensure test isolation. If you encounter test pollution: + +1. Verify each test properly cleans up resources +2. Check for static state leakage +3. Review test execution order + +### Debugging Integration Tests + +Enable debug logging by modifying `server/src/test/resources/logback-in-jvm-dtest.xml`: + +```xml + +``` + +### Performance Testing + +For performance-sensitive tests: + +1. Use `@Tag("heavy")` to categorize expensive tests +2. Monitor test execution times (logged automatically for tests ≥ 60 seconds) +3. Consider test parallelization settings + +### Environment-Specific Issues + +#### Docker Issues +- Ensure Docker daemon is running +- Verify Docker has sufficient resources allocated +- Check for port conflicts + +#### Java Version Issues +- Verify Java 11+ compatibility +- Check JVM arguments in CI environments +- Review module path configurations + +## Additional Resources + +- [Project README](README.md) - General project information +- [Contributing Guide](CONTRIBUTING.md) - Development contribution guidelines +- [CircleCI Configuration](.circleci/config.yml) - CI/CD pipeline details +- [Gradle Build Scripts](build.gradle) - Build configuration details + +For questions or issues not covered in this guide, please: +1. Check existing [JIRA issues](https://issues.apache.org/jira/projects/CASSANDRASC/issues/) +2. Join the discussion on [ASF Slack](https://s.apache.org/slack-invite) in #cassandra +3. Subscribe to the [mailing list](mailto:dev-subscribe@cassandra.apache.org) From c52e7e9b1454854e5b7123338a54fd69d91b5d53 Mon Sep 17 00:00:00 2001 From: Francisco Guerrero Date: Sat, 30 Aug 2025 14:39:14 -0700 Subject: [PATCH 2/6] Update TESTING.md Co-authored-by: Doug Rohrer <682821+JeetKunDoug@users.noreply.github.com> --- TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TESTING.md b/TESTING.md index 2af9042bd..18c52144b 100644 --- a/TESTING.md +++ b/TESTING.md @@ -37,7 +37,7 @@ This document provides a comprehensive guide to running tests for the Apache Cas ### Required Software 1. **Java 11 or higher** (OpenJDK or Oracle) -2. **Docker** for running integration tests +2. **Docker** for running integration tests that leverage the S3MockContainer to test the S3 client. Unless your test requires the S3 Mock Container, please create a unit or integration test. 3. **Git** for cloning repositories ### Build Prerequisites From 59eae1b9136098275f694edf695f31d065e1584f Mon Sep 17 00:00:00 2001 From: Francisco Guerrero Date: Sat, 30 Aug 2025 14:39:31 -0700 Subject: [PATCH 3/6] Update TESTING.md Co-authored-by: Doug Rohrer <682821+JeetKunDoug@users.noreply.github.com> --- TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TESTING.md b/TESTING.md index 18c52144b..92fd5d4b9 100644 --- a/TESTING.md +++ b/TESTING.md @@ -172,7 +172,7 @@ Integration tests support several environment variables: | Variable | Default | Description | |----------|---------|-------------| -| `INTEGRATION_MAX_HEAP_SIZE` | `3000M` | Maximum heap size for integration tests | +| `INTEGRATION_MAX_HEAP_SIZE` | `3000M` | Maximum heap size for integration tests, per parallel fork | | `INTEGRATION_MAX_PARALLEL_FORKS` | `4` | Number of parallel test forks | | `INTEGRATION_MTLS_ENABLED` | `true` | Enable mTLS for integration tests | From 7e25ac25078546491cc17cbbf808990342a45984 Mon Sep 17 00:00:00 2001 From: Francisco Guerrero Date: Sat, 30 Aug 2025 14:45:57 -0700 Subject: [PATCH 4/6] Address feedback from Doug --- TESTING.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/TESTING.md b/TESTING.md index 92fd5d4b9..11e615308 100644 --- a/TESTING.md +++ b/TESTING.md @@ -135,7 +135,11 @@ Cassandra instances. They are divided into two categories: ### Container Tests -Container-based tests run in Docker environments for additional isolation. +Container-based tests run in Docker environments for additional isolation. For example, +tests for the Restore from S3 feature leverage the testcontainer-based S3 service +(`com.adobe.testing.s3mock.testcontainers.S3MockContainer`). Only tests that require +testcontainer-like functionality should live under `src/containerTest/`; otherwise, +a unit or in JVM dtest integration test should suffice. **Location**: `src/containerTest/` in relevant modules @@ -170,11 +174,11 @@ Test fixtures provide shared test utilities and data across modules. Integration tests support several environment variables: -| Variable | Default | Description | -|----------|---------|-------------| -| `INTEGRATION_MAX_HEAP_SIZE` | `3000M` | Maximum heap size for integration tests, per parallel fork | -| `INTEGRATION_MAX_PARALLEL_FORKS` | `4` | Number of parallel test forks | -| `INTEGRATION_MTLS_ENABLED` | `true` | Enable mTLS for integration tests | +| Variable | Default | Description | +|----------------------------------|---------|------------------------------------------------------------| +| `INTEGRATION_MAX_HEAP_SIZE` | `3000M` | Maximum heap size for integration tests, per parallel fork | +| `INTEGRATION_MAX_PARALLEL_FORKS` | `4` | Number of parallel test forks | +| `INTEGRATION_MTLS_ENABLED` | `true` | Enable mTLS for integration tests | Example: ```bash From c69965474d2229a22e8284a0ac14f531704facb2 Mon Sep 17 00:00:00 2001 From: Francisco Guerrero Date: Tue, 30 Sep 2025 11:33:26 -0700 Subject: [PATCH 5/6] Apply suggestions from code review Co-authored-by: Doug Rohrer <682821+JeetKunDoug@users.noreply.github.com> --- TESTING.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/TESTING.md b/TESTING.md index 11e615308..dba372846 100644 --- a/TESTING.md +++ b/TESTING.md @@ -139,7 +139,7 @@ Container-based tests run in Docker environments for additional isolation. For e tests for the Restore from S3 feature leverage the testcontainer-based S3 service (`com.adobe.testing.s3mock.testcontainers.S3MockContainer`). Only tests that require testcontainer-like functionality should live under `src/containerTest/`; otherwise, -a unit or in JVM dtest integration test should suffice. +a unit or in JVM dtest integration test is preferred. **Location**: `src/containerTest/` in relevant modules @@ -228,7 +228,7 @@ Tests running on Java 11+ automatically receive optimized JVM arguments defined For faster development cycles: ```bash -# Run with code checks disabled +# Run the Sidecar server with code checks disabled. This is useful for local manual testing of endpoints. ./gradlew run # Run specific test class @@ -318,10 +318,12 @@ INTEGRATION_MAX_PARALLEL_FORKS=1 ./gradlew integrationTest Integration tests use `forkEvery = 1` to ensure test isolation. If you encounter test pollution: 1. Verify each test properly cleans up resources -2. Check for static state leakage -3. Review test execution order +2. Check for static state leakage within the failing test class. +3. Review test execution order. -### Debugging Integration Tests +If the test state pollution is unavoidable within the test class, it is reasonable to split the tests into multiple, independent test classes. + +### Enabling debug logging for Integration Tests Enable debug logging by modifying `server/src/test/resources/logback-in-jvm-dtest.xml`: From 95cd0e6f712a5d4d284363046063c0325ad16e16 Mon Sep 17 00:00:00 2001 From: Francisco Guerrero Date: Mon, 1 Dec 2025 05:31:30 -0800 Subject: [PATCH 6/6] Address review comments # Conflicts: # TESTING.md --- TESTING.md | 46 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/TESTING.md b/TESTING.md index dba372846..54b21d324 100644 --- a/TESTING.md +++ b/TESTING.md @@ -36,7 +36,7 @@ This document provides a comprehensive guide to running tests for the Apache Cas ### Required Software -1. **Java 11 or higher** (OpenJDK or Oracle) +1. **Java 11 or higher** 2. **Docker** for running integration tests that leverage the S3MockContainer to test the S3 client. Unless your test requires the S3 Mock Container, please create a unit or integration test. 3. **Git** for cloning repositories @@ -78,6 +78,39 @@ Create temporary aliases for nodes 2-20: for i in {2..20}; do sudo ip addr add "127.0.0.${i}/8" dev lo; done ``` +#### Hostname Configuration + +In addition to network aliases, you must add hostname entries to `/etc/hosts` for localhost1 through localhost20. + +Add the following entries to `/etc/hosts`: +``` +127.0.0.1 localhost1 +127.0.0.2 localhost2 +127.0.0.3 localhost3 +127.0.0.4 localhost4 +127.0.0.5 localhost5 +127.0.0.6 localhost6 +127.0.0.7 localhost7 +127.0.0.8 localhost8 +127.0.0.9 localhost9 +127.0.0.10 localhost10 +127.0.0.11 localhost11 +127.0.0.12 localhost12 +127.0.0.13 localhost13 +127.0.0.14 localhost14 +127.0.0.15 localhost15 +127.0.0.16 localhost16 +127.0.0.17 localhost17 +127.0.0.18 localhost18 +127.0.0.19 localhost19 +127.0.0.20 localhost20 +``` + +**Note**: Editing `/etc/hosts` requires sudo privileges: +```bash +sudo vi /etc/hosts +``` + ## Test Types ### Unit Tests @@ -154,7 +187,7 @@ Test fixtures provide shared test utilities and data across modules. ### Quick Test Commands ```bash -# Run all tests (unit + integration) +# Run all tests (unit + integration + container) ./gradlew check # Run only unit tests @@ -166,6 +199,9 @@ Test fixtures provide shared test utilities and data across modules. # Skip integration tests ./gradlew check -x integrationTest +# Skip container tests +./gradlew check -x containerTest + # Run with specific Cassandra versions ./gradlew test -Dcassandra.sidecar.versions_to_test=4.1,5.1 ``` @@ -331,9 +367,9 @@ Enable debug logging by modifying `server/src/test/resources/logback-in-jvm-dtes ``` -### Performance Testing +### Troubleshooting Long-Running Tests -For performance-sensitive tests: +For resource-intensive tests: 1. Use `@Tag("heavy")` to categorize expensive tests 2. Monitor test execution times (logged automatically for tests ≥ 60 seconds) @@ -349,7 +385,7 @@ For performance-sensitive tests: #### Java Version Issues - Verify Java 11+ compatibility - Check JVM arguments in CI environments -- Review module path configurations +- Review JVM options defined in [gradle/common/java11Options.gradle](gradle/common/java11Options.gradle) ## Additional Resources