From 6c9c45232539ccf1c5e75b230c3092fc727c58c4 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 31 Dec 2025 08:47:24 +0000 Subject: [PATCH 1/6] Initial plan From 584a8eb06a39efc6c90b972fea8abf3d34610bd3 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 31 Dec 2025 08:52:14 +0000 Subject: [PATCH 2/6] Add comprehensive integration tests analysis and TestContainers migration guide Co-authored-by: rrobetti <7221783+rrobetti@users.noreply.github.com> --- .../analysis/INTEGRATION_TESTS_ANALYSIS.md | 209 +++++ .../TESTCONTAINERS_MIGRATION_GUIDE.md | 732 ++++++++++++++++++ 2 files changed, 941 insertions(+) create mode 100644 documents/analysis/INTEGRATION_TESTS_ANALYSIS.md create mode 100644 documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md diff --git a/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md b/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md new file mode 100644 index 000000000..144cba4dc --- /dev/null +++ b/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md @@ -0,0 +1,209 @@ +# Integration Tests Analysis + +**Date:** December 31, 2024 +**Purpose:** Comprehensive analysis of all integration tests in the OJP project, identifying which tests use TestContainers and which do not. + +## Executive Summary + +- **Total Integration Tests:** 30 +- **Tests Using TestContainers:** 5 (SQL Server only) +- **Tests NOT Using TestContainers:** 25 (Oracle, DB2, PostgreSQL, MySQL, MariaDB, CockroachDB, H2, Multinode) + +## Tests Using TestContainers + +The following **5 tests** have been migrated to use TestContainers with SQL Server: + +### SQL Server Integration Tests + +| Test Class | Purpose | Uses TestContainers | +|-----------|---------|-------------------| +| `SQLServerBinaryStreamIntegrationTest.java` | Tests SQL Server-specific binary stream handling (VARBINARY, IMAGE types) | ✅ Yes | +| `SQLServerBlobIntegrationTest.java` | Tests SQL Server BLOB operations and large binary data | ✅ Yes | +| `SQLServerMultipleTypesIntegrationTest.java` | Tests multiple SQL Server data types | ✅ Yes | +| `SQLServerReadMultipleBlocksOfDataIntegrationTest.java` | Tests reading large result sets in SQL Server | ✅ Yes | +| `SqlServerXAIntegrationTest.java` | Tests XA distributed transactions in SQL Server | ✅ Yes | + +**Implementation Details:** +- Uses `@ArgumentsSource(SQLServerConnectionProvider.class)` annotation +- Uses `@EnabledIf("openjproxy.jdbc.testutil.SQLServerTestContainer#isEnabled")` for conditional execution +- Container: `mcr.microsoft.com/mssql/server:2022-latest` +- Shared singleton container across all tests for efficiency +- Automatic XA stored procedures installation +- Test user and database setup (`testuser`, `defaultdb`) + +**Key Files:** +- `SQLServerTestContainer.java` - Singleton container manager +- `SQLServerConnectionProvider.java` - JUnit ArgumentsProvider for dynamic connection details + +## Tests NOT Using TestContainers + +The following **25 tests** currently use CSV files for connection configuration and require external database instances: + +### Oracle Integration Tests (7 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `OracleBinaryStreamIntegrationTest.java` | `oracle_connections.csv` | Tests Oracle RAW and BLOB binary stream handling | +| `OracleBlobIntegrationTest.java` | `oracle_connections.csv` | Tests Oracle BLOB operations | +| `OracleMultipleTypesIntegrationTest.java` | `oracle_connections.csv` | Tests multiple Oracle data types | +| `OracleReadMultipleBlocksOfDataIntegrationTest.java` | `oracle_connections_with_record_counts.csv` | Tests reading large result sets in Oracle | +| `OracleXAIntegrationTest.java` | `oracle_xa_connection.csv` | Tests XA distributed transactions in Oracle | +| `BlobIntegrationTest.java` (partial) | `h2_mysql_mariadb_oracle_connections.csv` | Tests BLOB operations across multiple databases | +| `BasicCrudIntegrationTest.java` (partial) | `h2_postgres_mysql_mariadb_oracle_sqlserver_connections.csv` | Tests basic CRUD operations | + +**System Property:** `enableOracleTests=true/false` + +### DB2 Integration Tests (4 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `Db2BinaryStreamIntegrationTest.java` | `db2_connection.csv` | Tests DB2 binary stream handling | +| `Db2BlobIntegrationTest.java` | `db2_connection.csv` | Tests DB2 BLOB operations | +| `Db2MultipleTypesIntegrationTest.java` | `db2_connection.csv` | Tests multiple DB2 data types | +| `Db2ReadMultipleBlocksOfDataIntegrationTest.java` | `db2_connections_with_record_counts.csv` | Tests reading large result sets in DB2 | + +**System Property:** `enableDb2Tests=true/false` + +### PostgreSQL Integration Tests (4 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `PostgresMultipleTypesIntegrationTest.java` | `postgres_connection.csv` | Tests multiple PostgreSQL data types | +| `PostgresXAIntegrationTest.java` | `postgres_xa_connection.csv` | Tests XA distributed transactions in PostgreSQL | +| `BinaryStreamIntegrationTest.java` (partial) | `h2_postgres_connections.csv` | Tests binary stream handling | +| `ReadMultipleBlocksOfDataIntegrationTest.java` (partial) | `h2_postgres_connections_with_record_counts.csv` | Tests reading large result sets | + +**System Property:** `enablePostgresTests=true/false` + +### MySQL/MariaDB Integration Tests (4 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `MySQLMultipleTypesIntegrationTest.java` | `mysql_mariadb_connection.csv` | Tests multiple MySQL data types | +| `MySQLSpecificFeaturesIntegrationTest.java` | `mysql_mariadb_connection.csv` | Tests MySQL-specific features | +| `BlobIntegrationTest.java` (partial) | `h2_mysql_mariadb_oracle_connections.csv` | Tests BLOB operations | +| `BasicCrudIntegrationTest.java` (partial) | `h2_postgres_mysql_mariadb_oracle_sqlserver_connections.csv` | Tests basic CRUD operations | + +**System Properties:** +- `enableMySQLTests=true/false` +- `enableMariaDBTests=true/false` + +### CockroachDB Integration Tests (4 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `CockroachDBBinaryStreamIntegrationTest.java` | `cockroachdb_connection.csv` | Tests CockroachDB binary stream handling | +| `CockroachDBBlobIntegrationTest.java` | `cockroachdb_connection.csv` | Tests CockroachDB BLOB operations | +| `CockroachDBMultipleTypesIntegrationTest.java` | `cockroachdb_connection.csv` | Tests multiple CockroachDB data types | +| `CockroachDBReadMultipleBlocksOfDataIntegrationTest.java` | `cockroachdb_connection.csv` | Tests reading large result sets in CockroachDB | + +**System Property:** `enableCockroachDBTests=true/false` + +### H2 Integration Tests (5 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `H2MultipleTypesIntegrationTest.java` | `h2_connection.csv` | Tests multiple H2 data types | +| `BinaryStreamIntegrationTest.java` (partial) | `h2_postgres_connections.csv` | Tests binary stream handling | +| `BlobIntegrationTest.java` (partial) | `h2_mysql_mariadb_oracle_connections.csv` | Tests BLOB operations | +| `ReadMultipleBlocksOfDataIntegrationTest.java` (partial) | `h2_postgres_connections_with_record_counts.csv` | Tests reading large result sets | +| `BasicCrudIntegrationTest.java` (partial) | `h2_postgres_mysql_mariadb_oracle_sqlserver_connections.csv` | Tests basic CRUD operations | + +**System Property:** `enableH2Tests=true/false` + +### Multinode Integration Tests (2 tests) + +| Test Class | CSV File | Purpose | +|-----------|----------|---------| +| `MultinodeIntegrationTest.java` | `multinode_connection.csv` | Tests multi-node OJP server configuration | +| `MultinodeXAIntegrationTest.java` | `multinode_connection.csv` | Tests XA transactions across multiple OJP nodes | + +**Note:** These tests verify OJP multi-node functionality, not specific database features. + +### Other Integration Tests (1 test) + +| Test Class | Configuration | Purpose | +|-----------|--------------|---------| +| `MultiDataSourceIntegrationTest.java` | Hardcoded H2 URLs | Tests multi-datasource functionality with H2 in-memory databases | + +**System Property:** `enableH2Tests=true/false` (or runs by default when no other DB tests are enabled) + +## Database Coverage Summary + +| Database | Total Tests | Uses TestContainers | Uses CSV Files | TestContainers Available | +|----------|-------------|-------------------|---------------|------------------------| +| SQL Server | 5 | 5 | 0 | ✅ Yes (implemented) | +| Oracle | 7 | 0 | 7 | 🔶 Possible (commercial license required) | +| DB2 | 4 | 0 | 4 | 🔶 Possible (commercial license required) | +| PostgreSQL | 4 | 0 | 4 | ✅ Yes (easy migration) | +| MySQL | 4 | 0 | 4 | ✅ Yes (easy migration) | +| MariaDB | ~2 | 0 | ~2 | ✅ Yes (easy migration) | +| CockroachDB | 4 | 0 | 4 | ✅ Yes (easy migration) | +| H2 | 5 | 0 | 5 | ⚠️ N/A (embedded, no container needed) | +| Multinode | 2 | 0 | 2 | ⚠️ Special case (tests OJP architecture) | + +## Migration Priority Recommendations + +### High Priority (Easy & High Value) +1. **PostgreSQL** - 4 tests, excellent TestContainers support, widely used +2. **MySQL** - 4 tests, excellent TestContainers support, widely used +3. **MariaDB** - 2-4 tests, excellent TestContainers support, MySQL compatible + +### Medium Priority (Moderate Complexity) +4. **CockroachDB** - 4 tests, TestContainers support available, PostgreSQL compatible +5. **Oracle** - 7 tests, TestContainers support available but requires accepting license + +### Low Priority (Special Cases) +6. **DB2** - 4 tests, TestContainers support available but requires accepting license +7. **H2** - 5 tests, embedded database, TestContainers not beneficial +8. **Multinode** - 2 tests, require special OJP server setup, not database-specific + +## Current TestContainers Dependencies + +From `ojp-jdbc-driver/pom.xml`: + +```xml + + org.testcontainers + mssqlserver + 1.20.4 + test + +``` + +## Benefits of TestContainers Migration + +1. **No External Dependencies** - No need to maintain external database instances +2. **Consistency** - Same database version and configuration for all developers and CI/CD +3. **Isolation** - Each test run uses fresh, isolated database instances +4. **Speed** - Parallel test execution with isolated containers +5. **Simplicity** - Automatic container lifecycle management +6. **CI/CD Ready** - Easy integration with GitHub Actions and other CI systems +7. **Version Control** - Database version is defined in code, not documentation + +## Current Limitations + +### Tests Using CSV Files +- **Manual Setup Required** - Developers need to set up external databases +- **Configuration Drift** - Different developers may have different database versions/configurations +- **CI/CD Complexity** - Requires external database services or setup in CI pipelines +- **Resource Management** - Tests cannot clean up after themselves as easily +- **Documentation Overhead** - Need to maintain CSV files with connection details + +### CSV Configuration Files Location +All CSV files are located in: `ojp-jdbc-driver/src/test/resources/` + +Example CSV files: +- `oracle_connections.csv` +- `db2_connection.csv` +- `postgres_connection.csv` +- `mysql_mariadb_connection.csv` +- `cockroachdb_connection.csv` +- `h2_connection.csv` +- And various combined CSV files for multi-database tests + +## See Also + +- [TESTCONTAINERS_MIGRATION_GUIDE.md](./TESTCONTAINERS_MIGRATION_GUIDE.md) - Step-by-step guide to migrate tests to TestContainers +- [../SQLSERVER_TESTCONTAINER_GUIDE.md](../SQLSERVER_TESTCONTAINER_GUIDE.md) - SQL Server TestContainers implementation details +- [TestContainers Documentation](https://www.testcontainers.org/) - Official TestContainers documentation diff --git a/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md b/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md new file mode 100644 index 000000000..066bed921 --- /dev/null +++ b/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md @@ -0,0 +1,732 @@ +# TestContainers Migration Guide + +**Purpose:** Step-by-step guide for migrating integration tests from CSV-based configuration to TestContainers. + +## Table of Contents +- [Overview](#overview) +- [Prerequisites](#prerequisites) +- [Migration Steps](#migration-steps) +- [Database-Specific Guides](#database-specific-guides) +- [Best Practices](#best-practices) +- [Troubleshooting](#troubleshooting) + +## Overview + +This guide provides instructions for migrating OJP integration tests from external database connections (using CSV configuration files) to TestContainers. The SQL Server tests serve as the reference implementation. + +### Benefits of Migration +- ✅ No external database setup required +- ✅ Consistent test environment across all developers +- ✅ Faster test execution with container reuse +- ✅ Better CI/CD integration +- ✅ Automatic cleanup and isolation + +### Reference Implementation +See the SQL Server implementation as the gold standard: +- `SQLServerTestContainer.java` - Container management +- `SQLServerConnectionProvider.java` - JUnit integration +- `SQLServerBinaryStreamIntegrationTest.java` - Example test usage + +## Prerequisites + +### Required Tools +1. **Docker** - Must be installed and running + ```bash + docker --version + # Ensure Docker daemon is running + docker ps + ``` + +2. **Maven** - For building and testing + ```bash + mvn --version + ``` + +3. **Java 11+** - Required for running tests + ```bash + java -version + ``` + +### Required Knowledge +- Basic understanding of JUnit 5 +- Familiarity with Maven project structure +- Basic Docker concepts +- Understanding of JDBC connections + +## Migration Steps + +### Step 1: Add TestContainers Dependencies + +Add the appropriate TestContainers module to `ojp-jdbc-driver/pom.xml`: + +```xml + + + org.testcontainers + [container-module] + 1.20.4 + test + + + + + [jdbc-driver-group] + [jdbc-driver-artifact] + [jdbc-driver-version] + test + +``` + +**Available TestContainers Modules:** +| Database | Artifact ID | JDBC Driver Group | JDBC Driver Artifact | +|----------|------------|-------------------|---------------------| +| PostgreSQL | `postgresql` | `org.postgresql` | `postgresql` | +| MySQL | `mysql` | `com.mysql` | `mysql-connector-j` | +| MariaDB | `mariadb` | `org.mariadb.jdbc` | `mariadb-java-client` | +| Oracle | `oracle-xe` | `com.oracle.database.jdbc` | `ojdbc11` | +| CockroachDB | `cockroachdb` | `org.postgresql` | `postgresql` | + +### Step 2: Create Container Manager Class + +Create a singleton container manager in `ojp-jdbc-driver/src/test/java/openjproxy/jdbc/testutil/`: + +```java +package openjproxy.jdbc.testutil; + +import org.testcontainers.containers.[DatabaseContainer]; +import java.util.concurrent.locks.ReentrantLock; + +/** + * Singleton [Database] test container for all integration tests. + * Ensures all tests share the same container instance for efficiency. + */ +public class [Database]TestContainer { + + // Container image + private static final String [DATABASE]_IMAGE = "[image:tag]"; + + private static [DatabaseContainer] container; + private static boolean isStarted = false; + private static boolean shutdownHookRegistered = false; + private static ReentrantLock initLock = new ReentrantLock(); + + /** + * Gets or creates the shared test container instance. + * The container is automatically started on first access. + */ + public static [DatabaseContainer] getInstance() { + // Fast-path: if container already created and running, return it + [DatabaseContainer] local = container; + if (local != null && local.isRunning()) { + return local; + } + + initLock.lock(); + try { + if (container == null) { + container = new [DatabaseContainer]<>([DATABASE]_IMAGE); + // Add any database-specific configuration here + } + + if (!isStarted) { + container.start(); + isStarted = true; + + // Post-start initialization if needed + try { + // Database-specific setup (e.g., create users, databases, install extensions) + performPostStartSetup(); + } catch (Exception e) { + System.err.println("[[Database]TestContainer] Warning: Post-start setup failed: " + e.getMessage()); + } + + // Add shutdown hook + if (!shutdownHookRegistered) { + Runtime.getRuntime().addShutdownHook(new Thread(() -> { + if (container != null && container.isRunning()) { + container.stop(); + } + })); + shutdownHookRegistered = true; + } + } + + return container; + } finally { + initLock.unlock(); + } + } + + /** + * Database-specific post-start initialization. + */ + private static void performPostStartSetup() throws Exception { + // Example: Create test database, users, install extensions + // See SQLServerTestContainer for XA setup example + } + + /** + * Gets the JDBC URL for connecting to the test container. + */ + public static String getJdbcUrl() { + return getInstance().getJdbcUrl(); + } + + /** + * Gets the username for connecting to the test container. + */ + public static String getUsername() { + return getInstance().getUsername(); + } + + /** + * Gets the password for connecting to the test container. + */ + public static String getPassword() { + return getInstance().getPassword(); + } + + /** + * Checks if tests are enabled via system property. + */ + public static boolean isEnabled() { + return Boolean.parseBoolean(System.getProperty("enable[Database]Tests", "false")); + } +} +``` + +### Step 3: Create Connection Provider + +Create a JUnit ArgumentsProvider in the same package: + +```java +package openjproxy.jdbc.testutil; + +import org.jetbrains.annotations.NotNull; +import org.junit.jupiter.api.extension.ExtensionContext; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.ArgumentsProvider; +import java.util.stream.Stream; + +/** + * Custom ArgumentsProvider for [Database] integration tests. + * Provides connection details from TestContainers when tests are enabled. + */ +public class [Database]ConnectionProvider implements ArgumentsProvider { + + private static final String JDBC_PREFIX = "jdbc:"; + private static final String OJP_PROXY_HOST = System.getProperty("ojp.proxy.host", "localhost"); + private static final String OJP_PROXY_PORT = System.getProperty("ojp.proxy.port", "1059"); + private static final String OJP_PROXY_ADDRESS = OJP_PROXY_HOST + ":" + OJP_PROXY_PORT; + + @Override + public Stream provideArguments(ExtensionContext context) { + if (![Database]TestContainer.isEnabled()) { + return Stream.empty(); + } + + ConnectionProps props = getConnectionProps(); + return Stream.of( + Arguments.of(props.driverClass, props.ojpUrl, props.username, props.password) + ); + } + + private static @NotNull ConnectionProps getConnectionProps() { + // Initialize and start the TestContainer + [Database]TestContainer.getInstance(); + + // Get connection details + String containerJdbcUrl = [Database]TestContainer.getJdbcUrl(); + String username = [Database]TestContainer.getUsername(); + String password = [Database]TestContainer.getPassword(); + + // Build OJP JDBC URL + String driverClass = "org.openjproxy.jdbc.Driver"; + String urlWithoutPrefix = containerJdbcUrl.startsWith(JDBC_PREFIX) + ? containerJdbcUrl.substring(JDBC_PREFIX.length()) + : containerJdbcUrl; + + // Add any database-specific URL parameters here + + String ojpUrl = JDBC_PREFIX + "ojp[" + OJP_PROXY_ADDRESS + "]_" + urlWithoutPrefix; + + return new ConnectionProps(username, password, driverClass, ojpUrl); + } + + private static class ConnectionProps { + final String username; + final String password; + final String driverClass; + final String ojpUrl; + + ConnectionProps(String username, String password, String driverClass, String ojpUrl) { + this.username = username; + this.password = password; + this.driverClass = driverClass; + this.ojpUrl = ojpUrl; + } + } +} +``` + +### Step 4: Update Test Classes + +Update each integration test class to use the new provider: + +**Before (CSV-based):** +```java +@ParameterizedTest +@CsvFileSource(resources = "/database_connections.csv") +public void testMethod(String driverClass, String url, String user, String pwd) { + // Test code... +} +``` + +**After (TestContainers-based):** +```java +import openjproxy.jdbc.testutil.[Database]ConnectionProvider; +import org.junit.jupiter.api.condition.EnabledIf; +import org.junit.jupiter.params.provider.ArgumentsSource; + +@EnabledIf("openjproxy.jdbc.testutil.[Database]TestContainer#isEnabled") +public class [Database]SomeIntegrationTest { + + @ParameterizedTest + @ArgumentsSource([Database]ConnectionProvider.class) + public void testMethod(String driverClass, String url, String user, String pwd) { + // Test code remains the same... + } +} +``` + +### Step 5: Update Test Execution + +Run tests with the database test flag enabled: + +```bash +# Single database tests +mvn test -pl ojp-jdbc-driver -Denable[Database]Tests=true -Dtest="[Database]*" + +# All integration tests +mvn test -pl ojp-jdbc-driver -Denable[Database]Tests=true +``` + +## Database-Specific Guides + +### PostgreSQL Migration + +**Container Image:** `postgres:17-alpine` + +**Dependencies:** +```xml + + org.testcontainers + postgresql + 1.20.4 + test + +``` + +**Container Setup:** +```java +import org.testcontainers.containers.PostgreSQLContainer; + +container = new PostgreSQLContainer<>("postgres:17-alpine") + .withDatabaseName("testdb") + .withUsername("testuser") + .withPassword("testpass"); +``` + +**Tests to Migrate:** +- `PostgresMultipleTypesIntegrationTest.java` +- `PostgresXAIntegrationTest.java` +- `BinaryStreamIntegrationTest.java` (postgres-specific tests) +- `ReadMultipleBlocksOfDataIntegrationTest.java` (postgres-specific tests) + +### MySQL Migration + +**Container Image:** `mysql:8.4` + +**Dependencies:** +```xml + + org.testcontainers + mysql + 1.20.4 + test + +``` + +**Container Setup:** +```java +import org.testcontainers.containers.MySQLContainer; + +container = new MySQLContainer<>("mysql:8.4") + .withDatabaseName("testdb") + .withUsername("testuser") + .withPassword("testpass"); +``` + +**Tests to Migrate:** +- `MySQLMultipleTypesIntegrationTest.java` +- `MySQLSpecificFeaturesIntegrationTest.java` +- `BlobIntegrationTest.java` (mysql-specific tests) + +### MariaDB Migration + +**Container Image:** `mariadb:11` + +**Dependencies:** +```xml + + org.testcontainers + mariadb + 1.20.4 + test + +``` + +**Container Setup:** +```java +import org.testcontainers.containers.MariaDBContainer; + +container = new MariaDBContainer<>("mariadb:11") + .withDatabaseName("testdb") + .withUsername("testuser") + .withPassword("testpass"); +``` + +**Tests to Migrate:** +- Tests using `mysql_mariadb_connection.csv` (MariaDB-specific runs) + +### CockroachDB Migration + +**Container Image:** `cockroachdb/cockroach:latest-v24.3` + +**Dependencies:** +```xml + + org.testcontainers + cockroachdb + 1.20.4 + test + +``` + +**Container Setup:** +```java +import org.testcontainers.containers.CockroachContainer; + +container = new CockroachContainer("cockroachdb/cockroach:latest-v24.3") + .withDatabaseName("testdb") + .withUsername("testuser") + .withPassword("testpass"); +``` + +**Tests to Migrate:** +- `CockroachDBBinaryStreamIntegrationTest.java` +- `CockroachDBBlobIntegrationTest.java` +- `CockroachDBMultipleTypesIntegrationTest.java` +- `CockroachDBReadMultipleBlocksOfDataIntegrationTest.java` + +### Oracle Migration + +**Container Image:** `gvenzl/oracle-free:23-slim-faststart` + +**Dependencies:** +```xml + + org.testcontainers + oracle-free + 1.20.4 + test + +``` + +**Container Setup:** +```java +import org.testcontainers.containers.OracleContainer; + +container = new OracleContainer("gvenzl/oracle-free:23-slim-faststart") + .withDatabaseName("FREEPDB1") + .withUsername("testuser") + .withPassword("testpass"); +``` + +**Special Considerations:** +- Oracle containers are larger and slower to start +- Consider using shared containers across test classes +- May require accepting Oracle license terms + +**Tests to Migrate:** +- `OracleBinaryStreamIntegrationTest.java` +- `OracleBlobIntegrationTest.java` +- `OracleMultipleTypesIntegrationTest.java` +- `OracleReadMultipleBlocksOfDataIntegrationTest.java` +- `OracleXAIntegrationTest.java` + +## Best Practices + +### 1. Use Singleton Containers +Share a single container instance across all tests for a database to improve performance: +```java +private static [DatabaseContainer] container; // Shared instance +``` + +### 2. Implement Proper Locking +Use locks to prevent race conditions during container initialization: +```java +private static ReentrantLock initLock = new ReentrantLock(); +``` + +### 3. Add Shutdown Hooks +Ensure containers are properly stopped: +```java +Runtime.getRuntime().addShutdownHook(new Thread(() -> { + if (container != null && container.isRunning()) { + container.stop(); + } +})); +``` + +### 4. Use Conditional Test Execution +Enable/disable tests based on system properties: +```java +@EnabledIf("openjproxy.jdbc.testutil.[Database]TestContainer#isEnabled") +``` + +### 5. Handle Post-Start Initialization +Perform database-specific setup after container starts: +```java +private static void performPostStartSetup() throws Exception { + // Create users, databases, install extensions, etc. +} +``` + +### 6. Use Fast-Start Images +Choose container images optimized for testing: +- PostgreSQL: `postgres:17-alpine` +- MySQL: `mysql:8.4` +- Oracle: `gvenzl/oracle-free:23-slim-faststart` + +### 7. Clean Up Between Tests +Use `@BeforeEach` or transaction rollback to ensure test isolation: +```java +@BeforeEach +void setUp() { + // Clean up tables or use transactions +} +``` + +### 8. Configure Timeouts +Set appropriate timeouts for container startup: +```java +container.withStartupTimeout(Duration.ofMinutes(5)); +``` + +### 9. Add Logging +Include helpful logging for debugging: +```java +System.out.println("Testing with container URL: " + container.getJdbcUrl()); +``` + +### 10. Document Container Configuration +Add JavaDoc comments explaining container setup and requirements. + +## Advanced Topics + +### XA Transaction Support + +For XA-enabled tests (like `SqlServerXAIntegrationTest`), you need to install XA components: + +```java +private static void installXaSupport() throws Exception { + // Example for SQL Server + String[] cmd = { + "/opt/mssql-tools18/bin/sqlcmd", + "-S", "localhost", + "-U", getInstance().getUsername(), + "-P", getInstance().getPassword(), + "-d", "master", + "-C", + "-Q", "EXEC sp_sqljdbc_xa_install;" + }; + getInstance().execInContainer(cmd); +} +``` + +### Custom Network Configuration + +For multi-container tests: +```java +Network network = Network.newNetwork(); +container.withNetwork(network); +``` + +### Database Initialization Scripts + +Run SQL scripts on container startup: +```java +container.withInitScript("init.sql"); +``` + +### Reusable Containers + +For faster test execution across multiple Maven modules: +```java +container.withReuse(true); +``` + +## Troubleshooting + +### Issue: Container fails to start + +**Symptom:** Test fails with "Could not start container" + +**Solutions:** +1. Check Docker is running: `docker ps` +2. Check Docker has enough resources (memory, disk) +3. Try pulling image manually: `docker pull [image:tag]` +4. Check firewall/network settings +5. Enable TestContainers debugging: `-Dtestcontainers.debug=true` + +### Issue: Tests are slow + +**Symptom:** Tests take a long time to execute + +**Solutions:** +1. Use singleton containers (share across tests) +2. Use fast-start container images (alpine, slim variants) +3. Use container reuse: `container.withReuse(true)` +4. Run tests in parallel: `mvn -T 4 test` +5. Reduce startup timeouts appropriately + +### Issue: Port conflicts + +**Symptom:** "Port already in use" error + +**Solutions:** +1. TestContainers uses random ports by default +2. Ensure you're not hardcoding ports +3. Stop conflicting containers: `docker ps` and `docker stop [id]` + +### Issue: OJP server not running + +**Symptom:** "Connection refused" to localhost:1059 + +**Solutions:** +1. Start OJP server before running tests: + ```bash + java -jar ojp-server/target/ojp-server-*-shaded.jar & + ``` +2. Check server logs +3. Verify port 1059 is available + +### Issue: Database initialization fails + +**Symptom:** Post-start setup throws exceptions + +**Solutions:** +1. Add appropriate delays: `Thread.sleep(1000)` +2. Check container logs: `container.getLogs()` +3. Verify JDBC driver compatibility +4. Check database-specific requirements (e.g., Oracle license acceptance) + +### Issue: Tests work locally but fail in CI + +**Symptom:** Tests pass on developer machine but fail in GitHub Actions + +**Solutions:** +1. Ensure Docker is available in CI environment +2. Check CI has sufficient resources +3. Use matrix builds for testing multiple databases +4. Add container health checks +5. Increase timeouts in CI environment + +## CI/CD Integration + +### GitHub Actions Example + +```yaml +name: Database Integration Tests + +on: [push, pull_request] + +jobs: + test: + runs-on: ubuntu-latest + + strategy: + matrix: + database: [postgresql, mysql, mariadb, cockroachdb] + + steps: + - uses: actions/checkout@v4 + + - name: Set up JDK 21 + uses: actions/setup-java@v4 + with: + java-version: '21' + distribution: 'temurin' + cache: 'maven' + + - name: Build OJP Server + run: mvn clean package -pl ojp-server -DskipTests + + - name: Start OJP Server + run: | + java -jar ojp-server/target/ojp-server-*-shaded.jar & + sleep 10 + + - name: Run ${{ matrix.database }} Integration Tests + run: | + mvn test -pl ojp-jdbc-driver \ + -Denable${{ matrix.database }}Tests=true \ + -Dtest="${{ matrix.database }}*" + + - name: Upload Test Results + if: always() + uses: actions/upload-artifact@v4 + with: + name: test-results-${{ matrix.database }} + path: ojp-jdbc-driver/target/surefire-reports/ +``` + +## Migration Checklist + +Use this checklist when migrating a test class: + +- [ ] Add TestContainers dependency to `pom.xml` +- [ ] Add JDBC driver dependency (if not present) +- [ ] Create `[Database]TestContainer.java` class +- [ ] Create `[Database]ConnectionProvider.java` class +- [ ] Implement singleton pattern with proper locking +- [ ] Add shutdown hook for cleanup +- [ ] Implement post-start initialization if needed +- [ ] Update test class annotations: + - [ ] Add `@EnabledIf` annotation + - [ ] Replace `@CsvFileSource` with `@ArgumentsSource` + - [ ] Update import statements +- [ ] Test locally with container +- [ ] Update CI/CD pipeline if needed +- [ ] Document any database-specific requirements +- [ ] Remove or deprecate CSV configuration files +- [ ] Update relevant documentation + +## Additional Resources + +- [TestContainers Documentation](https://www.testcontainers.org/) +- [TestContainers Java Modules](https://www.testcontainers.org/modules/) +- [SQL Server TestContainers Guide](../SQLSERVER_TESTCONTAINER_GUIDE.md) +- [Integration Tests Analysis](./INTEGRATION_TESTS_ANALYSIS.md) +- [JUnit 5 Documentation](https://junit.org/junit5/docs/current/user-guide/) +- [Docker Documentation](https://docs.docker.com/) + +## Support + +If you encounter issues during migration: +1. Check this guide's troubleshooting section +2. Review the SQL Server implementation as reference +3. Check TestContainers documentation +4. Review existing GitHub issues +5. Create a new issue with detailed error messages and logs From 66e4febee29ef8569efc93df7fcf439a5d7460d1 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 31 Dec 2025 08:53:42 +0000 Subject: [PATCH 3/6] Add integration tests quick reference and update documentation index Co-authored-by: rrobetti <7221783+rrobetti@users.noreply.github.com> --- documents/README.md | 13 +- .../INTEGRATION_TESTS_QUICK_REFERENCE.md | 124 ++++++++++++++++++ 2 files changed, 136 insertions(+), 1 deletion(-) create mode 100644 documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md diff --git a/documents/README.md b/documents/README.md index 35a61d0e3..a585fcf26 100644 --- a/documents/README.md +++ b/documents/README.md @@ -44,7 +44,9 @@ Located in [configuration/](configuration/): - [OJP JDBC Configuration](configuration/ojp-jdbc-configuration.md) - JDBC driver configuration - [OJP Server Configuration](configuration/ojp-server-configuration.md) - Server configuration -## Database Setup Guides +## Testing + +### Database Setup Guides Located in [environment-setup/](environment-setup/): - [Run Local Databases](environment-setup/run-local-databases.md) - Quick setup for local testing @@ -53,6 +55,14 @@ Located in [environment-setup/](environment-setup/): - [Oracle Testing Guide](environment-setup/oracle-testing-guide.md) - [SQL Server Testing Guide](environment-setup/sqlserver-testing-guide.md) +### Integration Tests Analysis + +Located in [analysis/](analysis/): +- [Integration Tests Quick Reference](analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md) - Quick lookup table of all tests +- [Integration Tests Analysis](analysis/INTEGRATION_TESTS_ANALYSIS.md) - Comprehensive analysis of all integration tests +- [TestContainers Migration Guide](analysis/TESTCONTAINERS_MIGRATION_GUIDE.md) - Step-by-step guide for migrating tests to TestContainers +- [SQL Server TestContainer Guide](SQLSERVER_TESTCONTAINER_GUIDE.md) - SQL Server TestContainers implementation reference + ## Framework Integration Located in [java-frameworks/](java-frameworks/): @@ -141,6 +151,7 @@ All documentation is organized under the `documents/` folder with the following ``` documents/ ├── ADRs/ # Architecture Decision Records +├── analysis/ # Analysis documents and migration guides ├── code-contributions/ # Contributing guides ├── configuration/ # Configuration documentation ├── contributor-badges/ # Recognition program diff --git a/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md b/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md new file mode 100644 index 000000000..2303ca5cd --- /dev/null +++ b/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md @@ -0,0 +1,124 @@ +# Integration Tests Quick Reference + +Quick reference table for all integration tests in the OJP project. + +## Legend +- ✅ = Uses TestContainers +- ❌ = Uses CSV files (external database required) +- 🔧 = Ready to migrate to TestContainers +- 📝 = Special case / Not applicable for TestContainers + +## All Integration Tests (30 total) + +| # | Test Class | Database | Uses TestContainers | Migration Status | System Property | +|---|-----------|----------|-------------------|------------------|-----------------| +| 1 | `SQLServerBinaryStreamIntegrationTest.java` | SQL Server | ✅ | Complete | `enableSqlServerTests` | +| 2 | `SQLServerBlobIntegrationTest.java` | SQL Server | ✅ | Complete | `enableSqlServerTests` | +| 3 | `SQLServerMultipleTypesIntegrationTest.java` | SQL Server | ✅ | Complete | `enableSqlServerTests` | +| 4 | `SQLServerReadMultipleBlocksOfDataIntegrationTest.java` | SQL Server | ✅ | Complete | `enableSqlServerTests` | +| 5 | `SqlServerXAIntegrationTest.java` | SQL Server | ✅ | Complete | `enableSqlServerTests` | +| 6 | `OracleBinaryStreamIntegrationTest.java` | Oracle | ❌ 🔧 | Not migrated | `enableOracleTests` | +| 7 | `OracleBlobIntegrationTest.java` | Oracle | ❌ 🔧 | Not migrated | `enableOracleTests` | +| 8 | `OracleMultipleTypesIntegrationTest.java` | Oracle | ❌ 🔧 | Not migrated | `enableOracleTests` | +| 9 | `OracleReadMultipleBlocksOfDataIntegrationTest.java` | Oracle | ❌ 🔧 | Not migrated | `enableOracleTests` | +| 10 | `OracleXAIntegrationTest.java` | Oracle | ❌ 🔧 | Not migrated | `enableOracleTests` | +| 11 | `Db2BinaryStreamIntegrationTest.java` | DB2 | ❌ 🔧 | Not migrated | `enableDb2Tests` | +| 12 | `Db2BlobIntegrationTest.java` | DB2 | ❌ 🔧 | Not migrated | `enableDb2Tests` | +| 13 | `Db2MultipleTypesIntegrationTest.java` | DB2 | ❌ 🔧 | Not migrated | `enableDb2Tests` | +| 14 | `Db2ReadMultipleBlocksOfDataIntegrationTest.java` | DB2 | ❌ 🔧 | Not migrated | `enableDb2Tests` | +| 15 | `PostgresMultipleTypesIntegrationTest.java` | PostgreSQL | ❌ 🔧 | **High Priority** | `enablePostgresTests` | +| 16 | `PostgresXAIntegrationTest.java` | PostgreSQL | ❌ 🔧 | **High Priority** | `enablePostgresTests` | +| 17 | `BinaryStreamIntegrationTest.java` | PostgreSQL/H2 | ❌ 🔧 | **High Priority** | `enablePostgresTests` / `enableH2Tests` | +| 18 | `ReadMultipleBlocksOfDataIntegrationTest.java` | PostgreSQL/H2 | ❌ 🔧 | **High Priority** | `enablePostgresTests` / `enableH2Tests` | +| 19 | `MySQLMultipleTypesIntegrationTest.java` | MySQL | ❌ 🔧 | **High Priority** | `enableMySQLTests` | +| 20 | `MySQLSpecificFeaturesIntegrationTest.java` | MySQL | ❌ 🔧 | **High Priority** | `enableMySQLTests` | +| 21 | `BlobIntegrationTest.java` | MySQL/MariaDB/Oracle/H2 | ❌ 🔧 | **High Priority** | Multiple | +| 22 | `CockroachDBBinaryStreamIntegrationTest.java` | CockroachDB | ❌ 🔧 | Medium Priority | `enableCockroachDBTests` | +| 23 | `CockroachDBBlobIntegrationTest.java` | CockroachDB | ❌ 🔧 | Medium Priority | `enableCockroachDBTests` | +| 24 | `CockroachDBMultipleTypesIntegrationTest.java` | CockroachDB | ❌ 🔧 | Medium Priority | `enableCockroachDBTests` | +| 25 | `CockroachDBReadMultipleBlocksOfDataIntegrationTest.java` | CockroachDB | ❌ 🔧 | Medium Priority | `enableCockroachDBTests` | +| 26 | `H2MultipleTypesIntegrationTest.java` | H2 | ❌ 📝 | N/A (embedded) | `enableH2Tests` | +| 27 | `BasicCrudIntegrationTest.java` | Multiple DBs | ❌ 🔧 | Partial (SQL Server done) | Multiple | +| 28 | `MultiDataSourceIntegrationTest.java` | H2 | ❌ 📝 | N/A (special case) | `enableH2Tests` | +| 29 | `MultinodeIntegrationTest.java` | Multinode | ❌ 📝 | N/A (OJP architecture test) | N/A | +| 30 | `MultinodeXAIntegrationTest.java` | Multinode | ❌ 📝 | N/A (OJP architecture test) | N/A | + +## Summary by Database + +| Database | Total Tests | Using TestContainers | Using CSV | Migration Ready | +|----------|-------------|---------------------|-----------|-----------------| +| **SQL Server** | 5 | 5 (100%) | 0 | ✅ Complete | +| **Oracle** | 7 | 0 (0%) | 7 | 🔧 Ready | +| **DB2** | 4 | 0 (0%) | 4 | 🔧 Ready | +| **PostgreSQL** | 4 | 0 (0%) | 4 | 🔧 **High Priority** | +| **MySQL** | 4 | 0 (0%) | 4 | 🔧 **High Priority** | +| **MariaDB** | ~2 | 0 (0%) | ~2 | 🔧 **High Priority** | +| **CockroachDB** | 4 | 0 (0%) | 4 | 🔧 Medium Priority | +| **H2** | 5 | 0 (0%) | 5 | 📝 N/A (embedded) | +| **Multinode** | 2 | 0 (0%) | 2 | 📝 N/A (special) | + +## Migration Priority + +### 🔴 High Priority (Should migrate next) +1. **PostgreSQL** (4 tests) - Excellent TestContainers support, widely used +2. **MySQL** (4 tests) - Excellent TestContainers support, widely used +3. **MariaDB** (~2 tests) - Excellent TestContainers support, MySQL compatible + +### 🟡 Medium Priority +4. **CockroachDB** (4 tests) - Good TestContainers support, PostgreSQL compatible +5. **Oracle** (7 tests) - TestContainers support available, requires license acceptance + +### ⚪ Low Priority / Special Cases +6. **DB2** (4 tests) - TestContainers support available, requires license acceptance +7. **H2** (5 tests) - Embedded database, TestContainers not beneficial +8. **Multinode** (2 tests) - Tests OJP server architecture, not database-specific + +## CSV Configuration Files + +All CSV files are in: `ojp-jdbc-driver/src/test/resources/` + +| CSV File | Used By | Databases | +|----------|---------|-----------| +| `oracle_connections.csv` | Oracle tests | Oracle | +| `oracle_connections_with_record_counts.csv` | Oracle tests | Oracle | +| `oracle_xa_connection.csv` | Oracle XA tests | Oracle | +| `db2_connection.csv` | DB2 tests | DB2 | +| `db2_connections_with_record_counts.csv` | DB2 tests | DB2 | +| `postgres_connection.csv` | PostgreSQL tests | PostgreSQL | +| `postgres_xa_connection.csv` | PostgreSQL XA tests | PostgreSQL | +| `h2_postgres_connections.csv` | Cross-DB tests | H2, PostgreSQL | +| `h2_postgres_connections_with_record_counts.csv` | Cross-DB tests | H2, PostgreSQL | +| `mysql_mariadb_connection.csv` | MySQL/MariaDB tests | MySQL, MariaDB | +| `h2_mysql_mariadb_oracle_connections.csv` | Cross-DB tests | H2, MySQL, MariaDB, Oracle | +| `cockroachdb_connection.csv` | CockroachDB tests | CockroachDB | +| `h2_cockroachdb_connections.csv` | Cross-DB tests | H2, CockroachDB | +| `h2_connection.csv` | H2 tests | H2 | +| `sqlserver_connections.csv` | SQL Server tests (deprecated) | SQL Server | +| `sqlserver_xa_connection.csv` | SQL Server XA tests (deprecated) | SQL Server | +| `h2_postgres_mysql_mariadb_oracle_sqlserver_connections.csv` | Cross-DB tests | Multiple | +| `multinode_connection.csv` | Multinode tests | N/A | + +**Note:** SQL Server CSV files are now deprecated as all SQL Server tests use TestContainers. + +## Running Tests + +### With TestContainers (SQL Server) +```bash +# Requires Docker running +mvn test -pl ojp-jdbc-driver -DenableSqlServerTests=true -Dtest="SQLServer*" +``` + +### With CSV Files (External Database Required) +```bash +# Requires external database setup +mvn test -pl ojp-jdbc-driver -DenablePostgresTests=true -Dtest="Postgres*" +mvn test -pl ojp-jdbc-driver -DenableMySQLTests=true -Dtest="MySQL*" +mvn test -pl ojp-jdbc-driver -DenableOracleTests=true -Dtest="Oracle*" +# etc. +``` + +## See Also + +- [INTEGRATION_TESTS_ANALYSIS.md](./INTEGRATION_TESTS_ANALYSIS.md) - Detailed analysis +- [TESTCONTAINERS_MIGRATION_GUIDE.md](./TESTCONTAINERS_MIGRATION_GUIDE.md) - Step-by-step migration guide +- [../SQLSERVER_TESTCONTAINER_GUIDE.md](../SQLSERVER_TESTCONTAINER_GUIDE.md) - SQL Server reference implementation From 5c8818918642d9ce98aff996905e5b1139b72949 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 31 Dec 2025 08:54:55 +0000 Subject: [PATCH 4/6] Add analysis folder README with navigation guide Co-authored-by: rrobetti <7221783+rrobetti@users.noreply.github.com> --- documents/analysis/README.md | 149 +++++++++++++++++++++++++++++++++++ 1 file changed, 149 insertions(+) create mode 100644 documents/analysis/README.md diff --git a/documents/analysis/README.md b/documents/analysis/README.md new file mode 100644 index 000000000..0469ed4bf --- /dev/null +++ b/documents/analysis/README.md @@ -0,0 +1,149 @@ +# Integration Tests Analysis Documentation + +This folder contains comprehensive documentation about OJP's integration tests and their migration to TestContainers. + +## Documents Overview + +### 📋 [Integration Tests Quick Reference](INTEGRATION_TESTS_QUICK_REFERENCE.md) +**Quick lookup table for all integration tests** + +Best for: Quick reference, at-a-glance status check + +Contains: +- Complete table of all 30 integration tests +- TestContainers usage status for each test +- Migration priorities +- Database breakdown summary +- CSV configuration files mapping +- Test execution examples + +### 📊 [Integration Tests Analysis](INTEGRATION_TESTS_ANALYSIS.md) +**Comprehensive analysis of all integration tests** + +Best for: Understanding the full picture, planning migration efforts + +Contains: +- Executive summary with key statistics +- Detailed breakdown by database type (SQL Server, Oracle, DB2, PostgreSQL, MySQL, etc.) +- Individual test descriptions and purposes +- Implementation details for TestContainers tests +- Migration priority recommendations +- Current limitations and benefits comparison +- CSV configuration file locations + +### 📖 [TestContainers Migration Guide](TESTCONTAINERS_MIGRATION_GUIDE.md) +**Step-by-step guide for migrating tests to TestContainers** + +Best for: Actually performing the migration work + +Contains: +- Prerequisites and setup instructions +- Complete 5-step migration process +- Database-specific implementation guides: + - PostgreSQL (High Priority) + - MySQL (High Priority) + - MariaDB (High Priority) + - CockroachDB (Medium Priority) + - Oracle (Medium Priority) +- Code templates ready to use +- Best practices and patterns +- Advanced topics (XA, networking, initialization) +- Comprehensive troubleshooting section +- CI/CD integration examples +- Migration checklist + +## Quick Stats + +- **Total Integration Tests:** 30 +- **Using TestContainers:** 5 (16.7%) - SQL Server only +- **Using CSV Files:** 25 (83.3%) - All other databases +- **Migration Ready:** ~19 tests (PostgreSQL, MySQL, MariaDB, CockroachDB, Oracle, DB2) +- **Special Cases:** 6 tests (H2 embedded, Multinode tests) + +## Current State + +### ✅ Databases Using TestContainers +- **SQL Server** - 5 tests (100% migrated) + +### 🔧 Databases Ready for Migration +- **PostgreSQL** - 4 tests (High Priority) +- **MySQL** - 4 tests (High Priority) +- **MariaDB** - ~2 tests (High Priority) +- **CockroachDB** - 4 tests (Medium Priority) +- **Oracle** - 7 tests (Medium Priority, requires license) +- **DB2** - 4 tests (Low Priority, requires license) + +### 📝 Special Cases +- **H2** - 5 tests (Embedded database, TestContainers not applicable) +- **Multinode** - 2 tests (OJP architecture tests, not database-specific) + +## How to Use These Documents + +### If you're new to the project... +1. Start with **[Quick Reference](INTEGRATION_TESTS_QUICK_REFERENCE.md)** to see all tests at a glance +2. Read **[Analysis](INTEGRATION_TESTS_ANALYSIS.md)** for context and details +3. Review the SQL Server implementation as a reference + +### If you're planning to migrate tests... +1. Check **[Quick Reference](INTEGRATION_TESTS_QUICK_REFERENCE.md)** for migration priorities +2. Read **[Analysis](INTEGRATION_TESTS_ANALYSIS.md)** to understand the current state +3. Follow **[Migration Guide](TESTCONTAINERS_MIGRATION_GUIDE.md)** step-by-step +4. Reference the SQL Server implementation (see below) + +### If you're running tests... +1. Use **[Quick Reference](INTEGRATION_TESTS_QUICK_REFERENCE.md)** for test commands +2. Check the database-specific guides in `../environment-setup/` if using CSV files + +## Reference Implementation + +The SQL Server integration tests serve as the gold standard for TestContainers implementation: + +**Key Files:** +- `ojp-jdbc-driver/src/test/java/openjproxy/jdbc/testutil/SQLServerTestContainer.java` +- `ojp-jdbc-driver/src/test/java/openjproxy/jdbc/testutil/SQLServerConnectionProvider.java` +- `ojp-jdbc-driver/src/test/java/openjproxy/jdbc/SQLServerBinaryStreamIntegrationTest.java` + +**Documentation:** +- [SQL Server TestContainer Guide](../SQLSERVER_TESTCONTAINER_GUIDE.md) + +## Related Documentation + +- [Main Documentation Index](../README.md) +- [SQL Server TestContainer Guide](../SQLSERVER_TESTCONTAINER_GUIDE.md) +- [Database Setup Guides](../environment-setup/) +- [TestContainers Official Documentation](https://www.testcontainers.org/) + +## Benefits of TestContainers + +1. ✅ **No External Dependencies** - No need to set up external databases +2. ✅ **Consistency** - Same database version for all developers and CI +3. ✅ **Isolation** - Each test run uses fresh, isolated instances +4. ✅ **Speed** - Parallel test execution with container reuse +5. ✅ **Simplicity** - Automatic container lifecycle management +6. ✅ **CI/CD Ready** - Easy integration with GitHub Actions +7. ✅ **Version Control** - Database version defined in code + +## Current Limitations (CSV-based tests) + +1. ❌ **Manual Setup** - Developers must set up external databases +2. ❌ **Configuration Drift** - Different versions/configurations across environments +3. ❌ **CI/CD Complexity** - Requires external services or complex setup +4. ❌ **Resource Management** - Harder to clean up and ensure isolation +5. ❌ **Documentation Overhead** - Must maintain CSV files separately + +## Contributing + +To contribute to migration efforts: + +1. Pick a database from the **High Priority** list +2. Follow the **[Migration Guide](TESTCONTAINERS_MIGRATION_GUIDE.md)** +3. Test thoroughly locally +4. Update CI/CD workflows if needed +5. Submit a pull request with your changes + +## Questions? + +- Check the **[Migration Guide](TESTCONTAINERS_MIGRATION_GUIDE.md)** troubleshooting section +- Review the SQL Server reference implementation +- Consult the [TestContainers documentation](https://www.testcontainers.org/) +- Open an issue in the repository From 804d31d21a2921b4a8108c003bb238b296dc8bed Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 31 Dec 2025 09:07:20 +0000 Subject: [PATCH 5/6] Clarify Oracle licensing - Oracle Free/XE is free for testing Co-authored-by: rrobetti <7221783+rrobetti@users.noreply.github.com> --- documents/analysis/INTEGRATION_TESTS_ANALYSIS.md | 6 +++--- documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md | 4 ++-- documents/analysis/README.md | 4 ++-- documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md | 9 +++++---- 4 files changed, 12 insertions(+), 11 deletions(-) diff --git a/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md b/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md index 144cba4dc..1ca2c51cb 100644 --- a/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md +++ b/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md @@ -133,7 +133,7 @@ The following **25 tests** currently use CSV files for connection configuration | Database | Total Tests | Uses TestContainers | Uses CSV Files | TestContainers Available | |----------|-------------|-------------------|---------------|------------------------| | SQL Server | 5 | 5 | 0 | ✅ Yes (implemented) | -| Oracle | 7 | 0 | 7 | 🔶 Possible (commercial license required) | +| Oracle | 7 | 0 | 7 | ✅ Yes (Oracle Free/XE is free for testing) | | DB2 | 4 | 0 | 4 | 🔶 Possible (commercial license required) | | PostgreSQL | 4 | 0 | 4 | ✅ Yes (easy migration) | | MySQL | 4 | 0 | 4 | ✅ Yes (easy migration) | @@ -151,10 +151,10 @@ The following **25 tests** currently use CSV files for connection configuration ### Medium Priority (Moderate Complexity) 4. **CockroachDB** - 4 tests, TestContainers support available, PostgreSQL compatible -5. **Oracle** - 7 tests, TestContainers support available but requires accepting license +5. **Oracle** - 7 tests, TestContainers support available (Oracle Free/XE is free for testing) ### Low Priority (Special Cases) -6. **DB2** - 4 tests, TestContainers support available but requires accepting license +6. **DB2** - 4 tests, TestContainers support available but requires commercial license 7. **H2** - 5 tests, embedded database, TestContainers not beneficial 8. **Multinode** - 2 tests, require special OJP server setup, not database-specific diff --git a/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md b/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md index 2303ca5cd..2cc306dc5 100644 --- a/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md +++ b/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md @@ -66,10 +66,10 @@ Quick reference table for all integration tests in the OJP project. ### 🟡 Medium Priority 4. **CockroachDB** (4 tests) - Good TestContainers support, PostgreSQL compatible -5. **Oracle** (7 tests) - TestContainers support available, requires license acceptance +5. **Oracle** (7 tests) - Good TestContainers support (Oracle Free/XE is free for testing) ### ⚪ Low Priority / Special Cases -6. **DB2** (4 tests) - TestContainers support available, requires license acceptance +6. **DB2** (4 tests) - TestContainers support available, requires commercial license 7. **H2** (5 tests) - Embedded database, TestContainers not beneficial 8. **Multinode** (2 tests) - Tests OJP server architecture, not database-specific diff --git a/documents/analysis/README.md b/documents/analysis/README.md index 0469ed4bf..bf049c72b 100644 --- a/documents/analysis/README.md +++ b/documents/analysis/README.md @@ -70,8 +70,8 @@ Contains: - **MySQL** - 4 tests (High Priority) - **MariaDB** - ~2 tests (High Priority) - **CockroachDB** - 4 tests (Medium Priority) -- **Oracle** - 7 tests (Medium Priority, requires license) -- **DB2** - 4 tests (Low Priority, requires license) +- **Oracle** - 7 tests (Medium Priority, Oracle Free/XE is free for testing) +- **DB2** - 4 tests (Low Priority, requires commercial license) ### 📝 Special Cases - **H2** - 5 tests (Embedded database, TestContainers not applicable) diff --git a/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md b/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md index 066bed921..391715266 100644 --- a/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md +++ b/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md @@ -454,9 +454,10 @@ container = new OracleContainer("gvenzl/oracle-free:23-slim-faststart") ``` **Special Considerations:** -- Oracle containers are larger and slower to start -- Consider using shared containers across test classes -- May require accepting Oracle license terms +- Oracle containers are larger and slower to start (especially first time) +- Consider using shared containers across test classes for better performance +- **Oracle Free/XE is free to use for testing and development** - no license required +- Only production use of Oracle Database requires commercial licensing **Tests to Migrate:** - `OracleBinaryStreamIntegrationTest.java` @@ -630,7 +631,7 @@ container.withReuse(true); 1. Add appropriate delays: `Thread.sleep(1000)` 2. Check container logs: `container.getLogs()` 3. Verify JDBC driver compatibility -4. Check database-specific requirements (e.g., Oracle license acceptance) +4. Check database-specific requirements (e.g., container image availability) ### Issue: Tests work locally but fail in CI From 181be3a2993e9193d32f0cc471a3f56646e53015 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 31 Dec 2025 09:21:59 +0000 Subject: [PATCH 6/6] Clarify SQL Server and DB2 licensing - both have free editions for testing Co-authored-by: rrobetti <7221783+rrobetti@users.noreply.github.com> --- .../analysis/INTEGRATION_TESTS_ANALYSIS.md | 6 +- .../INTEGRATION_TESTS_QUICK_REFERENCE.md | 2 +- documents/analysis/README.md | 2 +- .../TESTCONTAINERS_MIGRATION_GUIDE.md | 55 +++++++++++++++++++ 4 files changed, 60 insertions(+), 5 deletions(-) diff --git a/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md b/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md index 1ca2c51cb..63e3c9d85 100644 --- a/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md +++ b/documents/analysis/INTEGRATION_TESTS_ANALYSIS.md @@ -132,9 +132,9 @@ The following **25 tests** currently use CSV files for connection configuration | Database | Total Tests | Uses TestContainers | Uses CSV Files | TestContainers Available | |----------|-------------|-------------------|---------------|------------------------| -| SQL Server | 5 | 5 | 0 | ✅ Yes (implemented) | +| SQL Server | 5 | 5 | 0 | ✅ Yes (implemented, Developer Edition free for testing) | | Oracle | 7 | 0 | 7 | ✅ Yes (Oracle Free/XE is free for testing) | -| DB2 | 4 | 0 | 4 | 🔶 Possible (commercial license required) | +| DB2 | 4 | 0 | 4 | ✅ Yes (Community Edition free for testing) | | PostgreSQL | 4 | 0 | 4 | ✅ Yes (easy migration) | | MySQL | 4 | 0 | 4 | ✅ Yes (easy migration) | | MariaDB | ~2 | 0 | ~2 | ✅ Yes (easy migration) | @@ -154,7 +154,7 @@ The following **25 tests** currently use CSV files for connection configuration 5. **Oracle** - 7 tests, TestContainers support available (Oracle Free/XE is free for testing) ### Low Priority (Special Cases) -6. **DB2** - 4 tests, TestContainers support available but requires commercial license +6. **DB2** - 4 tests, TestContainers support available (Community Edition free for testing) 7. **H2** - 5 tests, embedded database, TestContainers not beneficial 8. **Multinode** - 2 tests, require special OJP server setup, not database-specific diff --git a/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md b/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md index 2cc306dc5..42d6101e9 100644 --- a/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md +++ b/documents/analysis/INTEGRATION_TESTS_QUICK_REFERENCE.md @@ -69,7 +69,7 @@ Quick reference table for all integration tests in the OJP project. 5. **Oracle** (7 tests) - Good TestContainers support (Oracle Free/XE is free for testing) ### ⚪ Low Priority / Special Cases -6. **DB2** (4 tests) - TestContainers support available, requires commercial license +6. **DB2** (4 tests) - TestContainers support available (Community Edition free for testing) 7. **H2** (5 tests) - Embedded database, TestContainers not beneficial 8. **Multinode** (2 tests) - Tests OJP server architecture, not database-specific diff --git a/documents/analysis/README.md b/documents/analysis/README.md index bf049c72b..768bc2054 100644 --- a/documents/analysis/README.md +++ b/documents/analysis/README.md @@ -71,7 +71,7 @@ Contains: - **MariaDB** - ~2 tests (High Priority) - **CockroachDB** - 4 tests (Medium Priority) - **Oracle** - 7 tests (Medium Priority, Oracle Free/XE is free for testing) -- **DB2** - 4 tests (Low Priority, requires commercial license) +- **DB2** - 4 tests (Low Priority, Community Edition free for testing) ### 📝 Special Cases - **H2** - 5 tests (Embedded database, TestContainers not applicable) diff --git a/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md b/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md index 391715266..bf856f6b0 100644 --- a/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md +++ b/documents/analysis/TESTCONTAINERS_MIGRATION_GUIDE.md @@ -21,12 +21,28 @@ This guide provides instructions for migrating OJP integration tests from extern - ✅ Better CI/CD integration - ✅ Automatic cleanup and isolation +### Database Licensing for Testing + +**All major databases provide free editions for testing and development:** + +- **SQL Server Developer Edition** - Free for development and testing (not production) +- **Oracle Database Free/XE** - Free for development and testing (not production) +- **DB2 Community Edition** - Free for development and testing (not production) +- **PostgreSQL** - Open source, free for all use cases +- **MySQL** - Open source, free for all use cases +- **MariaDB** - Open source, free for all use cases +- **CockroachDB** - Open source core, free for all use cases + +**Important:** Production use of SQL Server, Oracle, and DB2 requires appropriate commercial licenses. TestContainers uses the free editions suitable for automated testing. + ### Reference Implementation See the SQL Server implementation as the gold standard: - `SQLServerTestContainer.java` - Container management - `SQLServerConnectionProvider.java` - JUnit integration - `SQLServerBinaryStreamIntegrationTest.java` - Example test usage +**Note:** SQL Server uses the Developer Edition container image (`mcr.microsoft.com/mssql/server:2022-latest`) which is free for development and testing. Production use requires appropriate licensing. + ## Prerequisites ### Required Tools @@ -466,6 +482,45 @@ container = new OracleContainer("gvenzl/oracle-free:23-slim-faststart") - `OracleReadMultipleBlocksOfDataIntegrationTest.java` - `OracleXAIntegrationTest.java` +### DB2 Migration + +**Container Image:** `icr.io/db2_community/db2:latest` + +**Dependencies:** +```xml + + org.testcontainers + db2 + 1.20.4 + test + +``` + +**Container Setup:** +```java +import org.testcontainers.containers.Db2Container; + +container = new Db2Container("icr.io/db2_community/db2:latest") + .withDatabaseName("testdb") + .withUsername("testuser") + .withPassword("testpass") + .acceptLicense(); // Required for DB2 Community Edition +``` + +**Special Considerations:** +- DB2 containers are large and can be slow to start (first time startup may take several minutes) +- **DB2 Community Edition is free to use for testing and development** - no license required +- You must accept the IBM license agreement using `.acceptLicense()` method +- Only production use of DB2 requires commercial licensing +- Consider using shared containers across test classes due to long startup time +- DB2 Community Edition has resource limits (16GB memory, 4 CPU cores) suitable for testing + +**Tests to Migrate:** +- `Db2BinaryStreamIntegrationTest.java` +- `Db2BlobIntegrationTest.java` +- `Db2MultipleTypesIntegrationTest.java` +- `Db2ReadMultipleBlocksOfDataIntegrationTest.java` + ## Best Practices ### 1. Use Singleton Containers