Skip to content

Conversation

diegomrsantos
Copy link
Member

@diegomrsantos diegomrsantos commented Sep 28, 2025

Issue Addressed

N/A - Performance improvement and code quality enhancement for test suite

Proposed Changes

In-Memory Database Migration

  • Migrate SQLite tests from file-based to in-memory databases for faster execution
  • Add explicit new_in_memory() API to replace magic :memory: string
  • Keep domain validation and restart simulation tests using file-based databases where persistence is required
  • Update slashing database tests in eth module to use in-memory databases

Test Fixture Refactoring

  • Eliminated operator generation duplication: Created generate_default_operators() helper to fix bug where operators were generated twice with different RSA keys
  • Removed legacy type alias: Deleted TestFixture type alias and updated all 38+ tests to use explicit InMemoryTestFixture/FileTestFixture for clarity
  • Arc-based database storage: Changed TestFixtureData.db to Arc<NetworkDatabase> to support tests requiring shared ownership (especially eth integration tests)
  • Created ProcessorFixture: New composite fixture combining InMemoryTestFixture with EventProcessor setup, eliminating awkward patterns in eth tests
  • Improved documentation: Added comprehensive comments explaining fixture state and design decisions

Additional Info

This PR improves both test performance and code quality:

  • Performance: In-memory databases eliminate temporary file I/O overhead
  • Correctness: Fixes operator generation bug that could cause subtle test failures
  • Maintainability: Cleaner fixture design with better composability and explicit types
  • Safety: Uses conditional compilation to ensure production code remains unchanged

All 38 database tests and 6 eth integration tests pass successfully.

Updates database and eth test modules to use in-memory SQLite databases
instead of temporary files for faster test execution and reduced I/O.

- Add test-only connection customizer for in-memory schema initialization
- Update TestFixture to use :memory: by default with file-based fallback
- Keep persistence-dependent tests using file-based databases
- Migrate slashing database tests to in-memory
- Use Option<TempDir> in TestFixture to avoid unnecessary temp directory creation for in-memory tests
- Change in-memory DB support from cfg(test) to feature="test-utils" to enable usage in integration tests
- Fix restart-related tests to use file-based fixtures for persistence validation
- Revert slashing database to use temp files (external crate limitation)
- Add documentation for schema migration bypass in in-memory tests
- Improve test comment clarity and remove redundant assertions
Renamed TestFixture constructor methods to better describe their behavior:
- new() → new_in_memory() - in-memory database with populated cluster
- new_empty() → new_in_memory_empty() - in-memory database, empty
- new_with_file() - kept as is - file-based database with populated cluster
- new_empty_with_file() - kept as is - file-based database, empty

Updated all call sites in database and eth integration tests.
@diegomrsantos diegomrsantos marked this pull request as ready for review October 15, 2025 15:42
@diegomrsantos diegomrsantos requested a review from dknopik October 15, 2025 22:09
@diegomrsantos diegomrsantos self-assigned this Oct 16, 2025
@diegomrsantos diegomrsantos added testing claude-recheck triggers claude review workflow to re-run labels Oct 16, 2025
Copy link
Member

@dknopik dknopik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, thanks!

Comment on lines 117 to 118
// Keep temp_dir alive by leaking it - tests are short-lived anyway
std::mem::forget(temp_dir);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes the test run to leave behind files. This might be irrelevant for CI runs, but I want my developer machine cleaned up.

Comment on lines 202 to 207
fn open_or_create(path: &Path, domain: DomainType) -> Result<Pool, DatabaseError> {
schema::ensure_up_to_date(path, domain)?;
Self::open_conn_pool(path)
if path.to_string_lossy() != ":memory:" {
schema::ensure_up_to_date(path, domain)?;
}
Self::open_conn_pool(path, domain)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not really like this or the conditional function below. Using the connection hook to create the schema also feels like a hack. What do you think about a separate function open_in_memory that creates a pool with a connection that does not time out and creates the schema always?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pushed a new whole design, let me know what you think

@diegomrsantos diegomrsantos added claude-recheck triggers claude review workflow to re-run and removed claude-recheck triggers claude review workflow to re-run labels Oct 16, 2025
@diegomrsantos diegomrsantos marked this pull request as draft October 16, 2025 21:11
…posability

Simplified test fixture design with the following improvements:

- Eliminated operator generation duplication by extracting generate_default_operators() helper
  This fixes a bug where operators were generated twice with different RSA keys

- Removed legacy TestFixture type alias and updated all tests to use explicit
  InMemoryTestFixture/FileTestFixture types for clarity

- Changed TestFixtureData.db to Arc<NetworkDatabase> to support tests requiring
  shared ownership (particularly eth integration tests)

- Created ProcessorFixture combining InMemoryTestFixture with EventProcessor setup,
  eliminating the pattern of creating fixtures just to extract the database

- Added comprehensive documentation explaining fixture state and design decisions

All 38 database tests and 6 eth integration tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@diegomrsantos diegomrsantos force-pushed the sqlite-in-memory-tests branch from 638fb8e to 0d79eeb Compare October 16, 2025 21:24
@diegomrsantos
Copy link
Member Author

@claude review this PR

Copy link

claude-code-actions-sigp bot commented Oct 16, 2025

Claude finished @diegomrsantos's task —— View job


Code Review In Progress

Todo List

  • Read core database changes (lib.rs, schema.rs, state.rs)
  • Review test fixture refactoring (tests/utils.rs)
  • Review eth test changes (eth/tests/common/mod.rs)
  • Analyze existing reviewer concerns
  • Evaluate overall approach and design
  • Provide comprehensive review feedback

Job RunBranch: sqlite-in-memory-tests

@diegomrsantos
Copy link
Member Author

@claude review this PR

Copy link

claude-code-actions-sigp bot commented Oct 17, 2025

Code Review In Progress

Todo List

  • Read core database changes (lib.rs, schema.rs, state.rs)
  • Review test fixture refactoring (tests/utils.rs)
  • Review eth test changes (eth/tests/common/mod.rs)
  • Analyze existing reviewer concerns
  • Evaluate overall approach and design
  • Provide comprehensive review feedback

Job RunBranch: sqlite-in-memory-tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-recheck triggers claude review workflow to re-run testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants