Docs/notebooks

[isabel-gomez-gs]

This PR contains the two jupyter notebooks that will be provided to clients together with the user documentation:

• sc-transformations-demo.ipynb contains the demo for interacting with Processors Controller endpoints.
• sc-rnaseq-demo.ipynb is the adapted version of the client demo for single-cell RNA-seq data analysis. The adaptations are mainly related to changes in the name of attributes.

[olegderyagin]

For proper version control, it is recommended to sync the notebook files between the odms-internal and user-docs. This could be done automatically via a "GitHub Actions" workflow placed in genestack/odms-internal/.github/workflows/sync-transformations-notebook.yml, which will trigger PR creation in user-docs upon changes in odms-internal.

name: Sync transformations-demo notebook to user-docs

on:
  push:
    branches: [develop]
    paths:
      - 'demo-materials/jupyter-notebook/transformations-demo/transformations-demo.ipynb'
      
concurrency:
  group: sync-transformations-notebook
  cancel-in-progress: true

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout private repo
        uses: actions/checkout@v4

      - name: Checkout public repo
        uses: actions/checkout@v4
        with:
          repository: genestack/user-docs
          token: ${{ secrets.USER_DOCS_PAT }}
          path: user-docs
          ref: develop

      - name: Copy and rename notebook
        run: |
          cp demo-materials/jupyter-notebook/transformations-demo/transformations-demo.ipynb \
             user-docs/docs/user-guide/doc-odm-user-guide/doc-odm-user-guide/notebooks/sc-transformations-demo.ipynb

      - name: Create PR if changed
        working-directory: user-docs
        env:
          GH_TOKEN: ${{ secrets.USER_DOCS_PAT }}
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"

          git add docs/user-guide/doc-odm-user-guide/doc-odm-user-guide/notebooks/sc-transformations-demo.ipynb

          if git diff --staged --quiet; then
            echo "No changes to sync."
            exit 0
          fi

          SRC_LOG=$(git -C "$GITHUB_WORKSPACE" log -1 --format="%h by %an: %s" -- \
            demo-materials/jupyter-notebook/transformations-demo/transformations-demo.ipynb)

          BRANCH="docs/notebooks-$(date +%Y%m%d-%H%M%S)"
          git checkout -b "$BRANCH"
          git commit -m "Sync transformations-demo notebook from odms-internal" \
                     -m "Source: $SRC_LOG"
          git push -u origin "$BRANCH"

          gh pr create \
            --repo genestack/user-docs \
            --base develop \
            --head "$BRANCH" \
            --title "Sync transformations-demo notebook" \
            --body "Automated sync from \`genestack/odms-internal\` (develop branch).

          Source: \`$SRC_LOG\`"

To make the workflow work, PAT permissions need to be set. I.e. someone with admin access to the genestack organization (or at least to both repos) needs to do two things:

1. Create a fine-grained PAT (Settings > Developer settings > Personal access tokens > Fine-grained tokens) with:

• Resource owner: genestack
• Repository access: genestack/user-docs only
• Permissions: Contents: Read and write, Pull requests: Read and write

2. Add it as a secret in genestack/odms-internal (Settings > Secrets and variables > Actions > New repository secret) named USER_DOCS_PAT.

P.S. For syncing the rna-seq notebook, it is better to create a separate workflow, e.g. genestack/odms-internal/.github/workflows/sync-rnaseq-notebook.yml

[olegderyagin]

Alternative to the files sync via "GitHub Actions" workflow, we could automate files copying and PR creation via a shell script, which has to be executed manually, but doesn't require PAT.

#!/usr/bin/env bash
set -euo pipefail

# ── Configuration ────────────────────────────────────────────────────
SRC_REPO="git@github.com:genestack/odms-internal.git"
SRC_BRANCH="develop"
SRC_DIR="demo-materials/jupyter-notebook/odm-trial/user-docs"

DST_REPO="git@github.com:genestack/user-docs.git"
DST_BRANCH="develop"
DST_DIR="docs/user-guide/doc-odm-user-guide/doc-odm-user-guide/notebooks"

PR_TITLE="Sync notebooks from odms-internal"
# ─────────────────────────────────────────────────────────────────────

TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT

printf "==> Cloning source repo (sparse, odms-internal @ $SRC_BRANCH)...\n"
git clone --depth 1 --branch "$SRC_BRANCH" --single-branch \
    --filter=blob:none --sparse \
    "$SRC_REPO" "$TMP/src"
git -C "$TMP/src" sparse-checkout set "$SRC_DIR"

printf "\n==> Cloning target repo (sparse, user-docs @ $DST_BRANCH)...\n"
git clone --depth 1 --branch "$DST_BRANCH" --single-branch \
    --filter=blob:none --sparse \
    "$DST_REPO" "$TMP/dst"
git -C "$TMP/dst" sparse-checkout set "$DST_DIR"

printf "\n==> Syncing $SRC_DIR -> $DST_DIR...\n"
mkdir -p "$TMP/dst/$DST_DIR"
rsync -av --delete "$TMP/src/$SRC_DIR/" "$TMP/dst/$DST_DIR/"

cd "$TMP/dst"

git add "$DST_DIR"

if git diff --staged --quiet; then
    echo "No changes to sync. Exiting."
    exit 0
fi

BRANCH="docs/sync-notebooks-$(date +%Y%m%d-%H%M%S)"
git checkout -b "$BRANCH"

git commit -m "Sync notebooks folder from odms-internal"

git push -u origin "$BRANCH"

gh pr create \
    --repo genestack/user-docs \
    --base "$DST_BRANCH" \
    --head "$BRANCH" \
    --title "$PR_TITLE" \
    --body "Automated sync of notebooks from \`genestack/odms-internal\` ($SRC_BRANCH branch)."

printf "\n==> Done.\n"

Prerequisites:

• gh CLI installed and authenticated (gh auth login).
• Git identity is set globally via git config --global user.name ... and git config --global user.email ...
• SSH access (or HTTPS with credentials) to both genestack/odms-internal and genestack/user-docs.
• rsync installed (pre-installed on macOS).
• Push access to genestack/user-docs (to push the branch) and permission to create PRs.

Usage:
Save it as sync_notebooks.sh anywhere convenient in the repo (e.g. at docs/notebooks/docs/user-guide/doc-odm-user-guide/doc-odm-user-guide/), make it executable (chmod +x sync_notebooks.sh), and run it whenever you want to sync.