TheWorldAvatar Migration: Project trimming, necessary upgrades, and spotlight ready preparations

[zhouxiaochi]

📝[Draft] Base Lib Cleanup: trimming, necessary upgrades, and spotlight ready preparations

Table of Contents

• 📝[Draft] Base Lib Cleanup: trimming, necessary upgrades, and spotlight ready preparations
  • Table of Contents
  • 1. Background
  • 2. Objectives
  • 3. Action Plan
    • 3.1. Scanning of the Old TWA Repository
    • 3.2. Dependency Mapping
    • 3.3. Test Coverage Analysis
    • 3.4. CI Workflow Verification
    • 3.5. Internal Dependency Analysis
    • 3.6. Code Trimming
    • 3.7. Iterative Dependency Removal
    • 3.8. Documentation Review
    • 3.9. Documentation Enhancement
  • 4. Current Status
  • 5. Timeline

1. Background

The BaseLib is one of the core building blocks of The World Avatar (TWA) project, providing essential functionalities used by many other components. The BaseLib has already been migrated from the old repository (cambridge-cares/TheWorldAvatar/JPS_BASE_LIB) to its own dedicated repository (TheWorldAvatar/baselib).

However, the BaseLib has accumulated technical debt over time:

1. The codebase has evolved through numerous iterations spanning several years of development

2. Throughout its lifetime, the project's design philosophy and objectives have undergone multiple major shifts

3. The project has seen contributions from multiple teams and individuals across different institutions:

   • Many key contributors have since moved on to other projects/organizations
   • Knowledge transfer between old and new team members has been challenging
   • Some components lack active maintainers after original developers' departure

4. As a result, parts of the BaseLib have become difficult to maintain:

   • Some code may no longer align with current project goals
   • Documentation and knowledge about certain components exists mainly as tribal knowledge

5. Several aspects of the BaseLib require significant updates:

   • Inconsistent development practices across different components
   • Documentation gaps due to contributor turnover
   • Potential outdated dependencies and frameworks

2. Objectives

To address these issues, we need to clean up and modernize the BaseLib with the following goals:

1. Conduct a comprehensive inventory scan of both the migrated BaseLib and the old TWA repository:

   • Map components in the old repository that depend on BaseLib functionality
   • Analyze the migrated BaseLib codebase to identify core components and their purposes
   • Document which classes and functions are actively used across both repositories
   • Create a dependency graph showing relationships between components
   • Identify any duplicate or redundant functionality between repositories
   • Track usage patterns to inform migration and cleanup decisions

2. Analyze and improve test coverage:

   • Assess the health of existing unit and integration tests
   • Identify gaps in test coverage
   • Create sub-issues for test improvements where needed

3. Trim the BaseLib to only include necessary components:

   • Remove unused or deprecated code
   • Clean up unnecessary dependencies
   • Ensure all remaining code is actively used and maintained

4. Upgrade components to modern development frameworks where needed:

   • Update to newer of Spring framework (4.x -> 6.x)
   • Update dependency versions when appropriate

5. Ensure proper documentation with minimal coverage:

   • General purpose of each component
   • Requirements for each component
   • Usage examples and guidelines
   • Both inline and file-based documentation

6. Prepare the BaseLib for sustainable long-term maintenance:

   • Comprehensive documentation
   • Robust test coverage
   • Clean dependency structure

3. Action Plan

3.1. Scanning of the Old TWA Repository

To understand the historical context and usage patterns of the BaseLib:

• Use GitHub API to analyze the old repository structure
• Identify all components that previously depended on the BaseLib
• Document the historical usage patterns to inform cleanup decisions

Responsible: @xiaochizhou

3.2. Dependency Mapping

Create a comprehensive map of dependencies:

• Identify which external projects depend on the BaseLib
• Document which specific classes and functions are used by external components
• Create a dependency graph to visualize relationships

Responsible: TBD

3.3. Test Coverage Analysis

Evaluate the current state of testing:

• Analyze existing unit and integration tests
• Measure test coverage using appropriate tools
• Identify critical components with insufficient test coverage
• Create sub-issues for test improvements where needed

Responsible: TBD

3.4. CI Workflow Verification

Ensure the continuous integration pipeline is robust:

• Review existing CI workflows in the new repository
• Verify that automated tests are running correctly
• Check that Dependabot is properly configured
• Make adjustments to CI configuration if needed

Responsible: TBD

3.5. Internal Dependency Analysis

Map the internal structure of the BaseLib:

• Analyze code-level dependencies between BaseLib components
• Identify tightly coupled components
• Document circular dependencies or other architectural issues
• Create a visualization of the internal dependency structure

Responsible: TBD

3.6. Code Trimming

Remove unnecessary code based on dependency analysis:

• Identify unused or deprecated code
• Remove components that are no longer needed
• Ensure all trimming operations pass the test suite
• Document all removed components for future reference

Responsible: TBD

3.7. Iterative Dependency Removal

Implement a careful process for removing dependencies:

• Establish a remove → test → adjust loop
• Remove one dependency at a time
• Run full test suite after each removal
• Adjust code as needed to maintain functionality
• Document all changes and their impacts

Responsible: TBD

3.8. Documentation Review

Assess the current state of documentation:

• Review inline code documentation
• Evaluate file-based documentation
• Check for README files and usage examples
• Identify documentation gaps and create sub-issues

Responsible: TBD

3.9. Documentation Enhancement

Improve documentation based on the review:

• Update inline documentation where needed
• Create or improve README files
• Add usage examples for key components
• Ensure documentation meets minimal requirements

Responsible: TBD

4. Current Status

To be updated... Estimated completion by Friday, April 11th, 2025 ... @xiaochizhou

5. Timeline

Phase	Description	Start Date	Estimated End Date	Status	Participants
1	Repository Scanning	2025-04-06	2025-04-11	In Progress	@xiaochizhou
2	Dependency Mapping	TBD	TBD	Not Started	TBD
3	Test Coverage Analysis	TBD	TBD	Not Started	TBD
4	CI Workflow Verification	TBD	TBD	Not Started	TBD
5	Internal Dependency Analysis	TBD	TBD	Not Started	TBD
6	Code Trimming	TBD	TBD	Not Started	TBD
7	Iterative Dependency Removal	TBD	TBD	Not Started	TBD
8	Documentation Review	TBD	TBD	Not Started	TBD
9	Documentation Enhancement	TBD	TBD	Not Started	TBD
10	Final Review	TBD	TBD	Not Started	TBD