[Task]: Update and improve documentation after repository restructure

[oehrlis]

Task Description

Update and improve all documentation to match the new repository structure, installation approach, and CI/test workflows. Ensure all user and contributor docs accurately reflect the latest structure, migration, and build/test/release process.

Task Type

Documentation

Affected Component

Documentation

Priority

Medium (Normal priority)

Motivation

Clear documentation is essential for a smooth migration from legacy layout to the new structure, as well as for onboarding users and contributors. This task addresses the risk of outdated docs, improves supportability, and ensures the repository matches best practices.

Implementation Plan

1. Review all current documentation for legacy layout dependencies and outdated instructions
2. Update README.md: new structure, build and install instructions, migration notes
3. Add/Revise docs/UPGRADE_FROM_OLD_LAYOUT.md detailing migration from legacy layout
4. Update CONTRIBUTING.md: describe new workflows for building, testing, contributing
5. Update in-line code comments in packaging/install/build scripts as needed
6. Add or update diagrams/images of new repo structure (in docs/images/ if needed)
7. Review all documentation for references to deprecated scripts/paths and revise
8. Ensure all installer, packaging, CI, and test workflow instructions are clear and accurate

Files to Modify

- README.md
- docs/UPGRADE_FROM_OLD_LAYOUT.md
- CONTRIBUTING.md
- docs/images/*
- In-line packaging/build/install script comments

Code Examples

Testing Approach

• Maniacally verify installer, build, and test docs work on a clean system
• Peer review of new/updated docs
• Confirm that migration from old layout is clear and works in a test
• Validate new contributors can build, test, and package from docs alone

Acceptance Criteria

• Implementation complete
• Documentation updated throughout repo
• All doc and workflow instructions tested
• Peer-reviewed/approved

Breaking Changes

No breaking changes

Dependencies

• Blocked by restructure-layout and installer migration tasks (sequential dependency)
• Related to: CI/CD and packaging workflows tasks
• Requires: Updated repo structure, installer, and test suite completed

Risks and Considerations

• Risk of missing references to old scripts or legacy layout
• Potential confusion for upgraders if docs aren't clear
• Needs regular review and peer review to catch outdated bits

Estimated Effort

No response

Subtasks

- [ ] Audit all documentation files
- [ ] Update README.md for structure, usage, migration
- [ ] Write or update docs/UPGRADE_FROM_OLD_LAYOUT.md
- [ ] Review and update CONTRIBUTING.md
- [ ] Update diagrams if needed
- [ ] Review inline script comments and references
- [ ] Peer review

Documentation Updates Required

• README.md
• docs/UPGRADE_FROM_OLD_LAYOUT.md
• CONTRIBUTING.md
• In-line packaging and installer script comments

Additional Notes

No response

Pre-submission Checklist

• I have searched existing issues to avoid duplicates
• I have provided a clear implementation plan
• I have defined acceptance criteria
• I have considered backward compatibility