Skip to content

Support creating a local JupyterLite deployment with git2cpp built from the local repo #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,11 @@ The CLI is tested using `python`. From the top-level directory:
```bash
pytest -v
```

# JupyterLite deployment

The `lite-deploy` directory contains everything needed to build the local `git2cpp` source code as
an [Emscripten-forge](https://emscripten-forge.org/) package and create a local JupyterLite
deployment that can run it in a [terminal](https://github.com/jupyterlite/terminal).

See the `README.md` in the `lite-deploy` directory for further details.
6 changes: 6 additions & 0 deletions lite-deploy/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.cockle_temp/
.jupyterlite.doit.db
cockle-config.json
cockle_wasm_env/
dist/
em-forge-recipes/
40 changes: 40 additions & 0 deletions lite-deploy/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
default: all
.PHONY: all build-deployment build-recipe clean clean-deployment clean-package modify-recipe rebuild serve

EM_FORGE_RECIPES_DIR = em-forge-recipes
GIT2CPP_RECIPE_DIR = recipes/recipes_emscripten/git2cpp
BUILT_PACKAGE_DIR = $(EM_FORGE_RECIPES_DIR)/output

# Note removing the .git directory otherwise `git clean -fxd` will not remove the directory.
$(EM_FORGE_RECIPES_DIR):
git clone https://github.com/emscripten-forge/recipes --depth 1 $@
rm -rf $@/.git

modify-recipe: $(EM_FORGE_RECIPES_DIR)
python modify-recipe.py $(EM_FORGE_RECIPES_DIR)/$(GIT2CPP_RECIPE_DIR)

build-recipe: modify-recipe
cd $(EM_FORGE_RECIPES_DIR) && pixi run build-emscripten-wasm32-pkg $(GIT2CPP_RECIPE_DIR)

build-deployment: build-recipe
jupyter lite --version
COCKLE_WASM_EXTRA_CHANNEL=./$(BUILT_PACKAGE_DIR) jupyter lite build --output-dir dist

all: build-deployment

# Rebuild package and deployment after changing git2cpp source code.
rebuild: clean-package all

# Run `make` before this.
serve:
npx static-handler dist

clean: clean-deployment
rm -rf $(EM_FORGE_RECIPES_DIR)

clean-package:
rm -rf $(BUILT_PACKAGE_DIR) cockle_wasm_env/

# Clean the deployment without removing the built package.
clean-deployment:
rm -rf .cockle_temp/ .jupyterlite.doit.db cockle-config.json cockle_wasm_env/ dist/
49 changes: 49 additions & 0 deletions lite-deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# JupyterLite deployment

This directory contains everything needed to build the local `git2cpp` source code as an
[Emscripten-forge](https://emscripten-forge.org/) package and create a local JupyterLite deployment
that can run it in a [terminal](https://github.com/jupyterlite/terminal).

It works on Linux and macOS but not Windows.

It uses a separate `micromamba` environment defined in `deploy-environment.yml`. To set this up use:
```bash
micromamba create -f deploy-environment.yml
micromamba activate git2cpp-deploy
```

Then to build `git2cpp` and create the JupyterLite deployment use:
```bash
make
```

This performs the following steps:

1. Clones the `emscripten-forge/recipes` repository.
2. Modifies the `git2cpp` recipe (using `modify_recipe.py`) to build from the local `git2cpp` source code in `../src/`.
3. Builds the package from the recipe using `pixi`.
4. Builds the JupyterLite deployment in the `dist/` directory using the locally-built package.

The built package will be in the `em-forge-recipes/output/emscripten-wasm32` directory with a name
something like `git2cpp-0.0.3-h2072262_3.tar.bz2`. If you compare this with the latest
Emscripten-forge package on `https://prefix.dev/channels/emscripten-forge-dev/packages/git2cpp`,
the local package should have the same version number and the build number should be one higher.

To serve the JupyterLite deployment use:
```bash
make serve
```
and open a web browser at the URL http://localhost:8080/. Start a terminal and run the
`cockle-config` command (typing `coc`, then the tab key then enter should suffice). Amongst the
displayed information should be the `git2cpp` package showing that it is from a local directory
such as `file:///something-or-other/git2cpp/lite-deploy/em-forge-recipes/output` rather than from
`prefix.dev` such as `https://repo.prefix.dev/emscripten-forge-dev`.

After making changes to `git2cpp` source code, to rebuild the package and deployment use:
```bash
make rebuild
```
and then re-serve using:
```bash
make serve
```
19 changes: 19 additions & 0 deletions lite-deploy/cockle-config-in.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"packages": {
"git2cpp": {},
"nano": {},
"tree": {},
"vim": {}
},
"aliases": {
"git": "git2cpp",
"vi": "vim"
},
"environment": {
"GIT_CORS_PROXY": "https://corsproxy.io/?url=",
"GIT_AUTHOR_NAME": "Jane Doe",
"GIT_AUTHOR_EMAIL": "jane.doe@blabla.com",
"GIT_COMMITTER_NAME": "Jane Doe",
"GIT_COMMITTER_EMAIL": "jane.doe@blabla.com"
}
}
14 changes: 14 additions & 0 deletions lite-deploy/deploy-environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: git2cpp-deploy
channels:
- conda-forge
dependencies:
# To modify emscripten-forge recipe
- python
- pyyaml
# To build emscripten-forge recipe
- pixi
- rattler-build
# For JupyterLite deployment
- jupyter_server
- jupyterlite-terminal
- nodejs
6 changes: 6 additions & 0 deletions lite-deploy/jupyter-lite.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"jupyter-lite-schema-version": 0,
"jupyter-config-data": {
"terminalsAvailable": true
}
}
49 changes: 49 additions & 0 deletions lite-deploy/modify-recipe.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Modify the git2cpp emscripten-forge recipe to build from the local repo.
# This can be called repeatedly and will produce the same output.

from pathlib import Path
import shutil
import sys
import yaml


def quit(msg):
print(msg)
exit(1)

if len(sys.argv) < 2:
quit(f'Usage: {sys.argv[0]} <recipe directory containing yaml file to modify>')

input_dir = Path(sys.argv[1])
if not input_dir.is_dir():
quit(f'{input_dir} should exist and be a directory')

input_filename = input_dir / 'recipe.yaml'
if not input_filename.is_file():
quit(f'{input_filename} should exist and be a file')

# If backup does not exist create it.
input_backup = input_dir / 'recipe_original.yaml'
backup_exists = input_backup.exists()
if not backup_exists:
shutil.copy(input_filename, input_backup)

# Read and parse input backup file which is the original recipe file.
with open(input_backup) as f:
recipe = yaml.safe_load(f)

build_number = recipe['build']['number']
print(f' Changing build number from {build_number} to {build_number+1}')
recipe['build']['number'] = build_number+1

source = recipe['source']
if not ('sha256' in source and 'url' in source):
raise RuntimeError('Expected recipe to have both a source sha256 and url')
del source['sha256']
del source['url']
print(' Changing source to point to local git2cpp repo')
source['path'] = '../../../../../'

# Overwrite recipe file.
with open(input_filename, 'w') as f:
yaml.dump(recipe, f)