Migrate resource hub to svelte

[rkdan]

Migrate Resource Hub to a Centralized SvelteKit Monorepo

Issue Description:

1. Background & Current Situation

Our current Resource Hub aggregates workshop content from multiple individual repositories. The workflow involves:

• Workshop content (primarily Markdown files and Jupyter notebooks) residing in separate Git repositories.
• A central Resource Hub repository that uses GitHub Actions (or a similar CI/CD process) to:
  • Fetch the latest content from each workshop repository.
  • Combine all content.
  • Build a static website using "Material for MkDocs".
• Whenever a workshop repository is updated, a trigger (e.g., webhook, repository dispatch) initiates a rebuild of the entire Resource Hub website.

2. Why This Approach Was Chosen (and its Pros)

• Ease of Workshop Updates: Lecturers/content creators can update their specific workshop content in a familiar, isolated environment without needing to understand the entire hub's build process.
• Markdown-centric: Content is primarily in Markdown, which is easy to write and learn.
• Jupyter Notebook Rendering: MkDocs (with appropriate plugins) handles the conversion of Jupyter notebooks into web-viewable formats reasonably well.

3. Identified Problems & Motivation for Change (Cons)

The current multi-repo setup with MkDocs, while functional, presents several growing challenges:

• Build Process:
  • Slow Build Times: Rebuilding the entire site for a single workshop update is inefficient, leading to slower deployments (though currently "not terrible," this will worsen as content grows).
  • Occasional Build Failures: The distributed nature and multiple points of data fetching can lead to intermittent build failures, making the process less reliable.
  • Complexity ("Messy"): Managing dependencies, fetch logic, and potential conflicts across multiple repos and a central build script is becoming cumbersome.
• User Experience & Aesthetics:
  • Poor Website Navigation: The current MkDocs structure may not offer the most intuitive or flexible navigation experience for a diverse range of resources.
  • "Looks Cheap": Material for MkDocs, while excellent for documentation, can give a generic "docs" feel. We aim for a more polished, custom "website" experience for the Resource Hub.
• Technical & Operational Debt:
  • Decentralized Cookies & Analytics: Managing cookie consent and website analytics (e.g., Google Analytics, Plausible) consistently across what is perceived as a single hub becomes difficult. Each workshop potentially acts as its own mini-site rather than a unified platform.
  • Extraneous Content in Workshop Repos: Workshop repositories often contain build configurations, MkDocs specific files, or other content that is only relevant to the old Resource Hub build, not the workshop's core educational material. This adds clutter.

4. Proposed Solution: Centralized SvelteKit Monorepo

We propose migrating the entire Resource Hub to a single, centralized SvelteKit repository.

• Content Migration:
  • All existing workshop Markdown files and Jupyter notebooks will be moved into structured directories within this new SvelteKit application.
  • The existing individual workshop repositories will be streamlined to only contain code, datasets, or other assets directly used within the workshop exercises (e.g., Python scripts, Dockerfiles for the workshop environment, sample data). They will no longer contain the Markdown/documentation content that forms the narrative of the workshop on the Resource Hub.
• Technology Stack:
  • SvelteKit: A modern web framework that allows for server-side rendering (SSR) or static site generation (SSG), providing flexibility and performance.
  • Markdown & Jupyter Notebook Rendering: Leverage Svelte components and existing libraries (as explored in the svelte-migration branch) to render these formats consistently and beautifully within the SvelteKit application.

5. Key Objectives & Success Criteria

• Improved Performance: Faster build times and page loads.
• Enhanced Reliability: More stable and predictable build and deployment process.
• Unified User Experience: Consistent navigation, styling, and branding across all workshop content.
• Centralized Analytics & Cookies: Single point of management for analytics and cookie consent.
• Simplified Maintenance: Easier to manage dependencies, updates, and the overall codebase from a single repository.
• Developer-Friendly: A modern, enjoyable development experience for those working on the Resource Hub.
• Clean Separation of Concerns: Workshop repos contain only workshop-specific code/assets; the SvelteKit repo handles all presentation and content rendering.

6. Foreseen Challenges & Areas to Investigate

• Content Structure within SvelteKit:
  • How should the diverse workshop materials be organized within the SvelteKit project's directory structure?
  • Consider workshops that are:
    • Walkthroughs/Tutorials (e.g., Docker, Packaging/Publishing).
    • Notebook-heavy with code packages & slides (e.g., Hands-on LLMs, Diffusion Models).
    • Purely presentation-based (e.g., LLMs and GenAI slides - how will these be integrated? Convert to Markdown? Embed? Link?).
• Consistent Rendering Logic:
  • Ensure Markdown (including any specific flavors or extensions currently used, e.g., admonitions, tables, syntax highlighting) renders correctly and consistently.
  • Ensure Jupyter Notebooks (.ipynb files) are parsed and rendered into attractive, interactive (if applicable) Svelte components. This includes code cells, outputs, images, and LaTeX.
• Integration of Other Key Pages:
  • Software Page: This page should dynamically list or link to software developed by Accelerate. How will this information be sourced and displayed? (e.g., from a separate data file, API, manually curated).
  • Community Page: Define the content and structure for the community page.
  • General Site Navigation: Design an intuitive top-level and sub-level navigation structure.
• URL/Redirect Strategy: Plan how old URLs will redirect to new ones to avoid breaking links.
• Asset Management: How will images, downloadable files, and other assets associated with workshops be handled in the new structure?

7. Starting Point & Initial Tasks

• Familiarize yourself with the svelte-migration branch:
  • Understand the proof-of-concept methods for rendering Markdown and Jupyter notebooks.
  • Task: Refactor this rendering logic to be globally accessible and reusable across different parts of the SvelteKit application (e.g., create utility functions, Svelte components, or services).
• Content Inventory & Mapping:
  • Task: Create a spreadsheet or document listing all current workshop repositories and their content types (Markdown, notebooks, slides, code).
  • Task: For each workshop, decide how its content will be migrated and structured in the new SvelteKit application.
• Basic SvelteKit Application Structure:
  • Task: Set up the basic routing for a few key workshop pages and the "Software" and "Community" pages.
  • Task: Implement a simple navigation bar and footer.
• Proof of Concept - Migrate One Workshop:
  • Task: Choose one simple workshop (e.g., a predominantly Markdown-based one) and migrate its content into the SvelteKit app.
  • Task: Ensure its Markdown files render correctly using the global rendering logic.
  • Task: Update the original workshop's repository to remove the migrated content, leaving only relevant code/assets.

8. Tools & Technologies to Learn/Use

• Svelte & SvelteKit
• JavaScript/TypeScript
• Markdown (and potentially remark/rehype ecosystem for processing)
• Jupyter Notebook format (understanding its JSON structure might be helpful)
• Git & GitHub
• Basic HTML/CSS

9. Questions to Consider & Discuss

• What is the desired look and feel for the new Resource Hub? Are there any example websites we like?
• How should versioning of workshop content be handled in the new monorepo?
• What level of interactivity is required for the rendered Jupyter notebooks (e.g., do code cells need to be executable live, or is static rendering sufficient)?
• What are the specific requirements for the "Software" and "Community" pages?
• How will we handle deployment of the new SvelteKit application?