Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
faa9c00
Added Dev Container configuration and updated preferred Python version.
wolfmanstout Dec 17, 2023
d3d0848
Manually forward port 8000 (auto-forwarding isn't working on GitHub C…
wolfmanstout Dec 17, 2023
f2089ef
Put OZtree_original into a shared volume for ease of access.
wolfmanstout Dec 21, 2023
371341b
Improve Dev Container setup docs.
wolfmanstout Dec 25, 2023
09c7fb5
Add optional auth table step to the Dev Container installation.
wolfmanstout Dec 25, 2023
b321929
Improve comments in devcontainer.json.
wolfmanstout Jan 2, 2024
ea88331
Revert preferred_python3 (will apply this change separately).
wolfmanstout Jan 16, 2024
58899c2
Update to Python 3.10.
wolfmanstout Jan 17, 2024
f772769
Update comment to make it clear that the web2py versions should match…
wolfmanstout Jan 18, 2024
6e89e07
Publish devcontainer server on 8080 and improve documentation.
wolfmanstout Jan 20, 2024
723b0c5
Map port 3306 to 3307 for local MySQL debugging.
wolfmanstout Jan 21, 2024
d576f1b
Document port 3307.
wolfmanstout Jan 21, 2024
5a1ac68
Improved dev container docs, updated to work with latest changes, mov…
wolfmanstout Feb 7, 2024
f1a8d89
Merge remote-tracking branch 'upstream/main' into dev_container
wolfmanstout Feb 7, 2024
b940862
Update docker-compose.yml source paths.
wolfmanstout Feb 7, 2024
ef4361e
Improved container comments and docs.
wolfmanstout Feb 10, 2024
4264008
Don't suggest setting is_testing to False in dev container instructions.
wolfmanstout Feb 10, 2024
6af9b4b
Merge branch 'main' into dev_container
hyanwong Mar 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .devcontainer/dev.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM mcr.microsoft.com/devcontainers/javascript-node:18
WORKDIR /opt
# Should align with https://github.com/OneZoom/OZtree-docker/blob/main/Dockerfile
RUN git clone --recursive https://github.com/web2py/web2py.git --depth 1 --branch v2.27.1 --single-branch web2py \
&& chown -R node:node web2py
# Required to avoid build issue when running grunt
ENV NODE_OPTIONS=--openssl-legacy-provider=0
ENV PATH=${PATH}:/opt/web2py/applications/OZtree/node_modules/.bin
24 changes: 24 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// For format details, see https://aka.ms/devcontainer.json.
{
"name": "OZTree Dev",
"dockerComposeFile": "docker-compose.yml",
"service": "dev",
"workspaceFolder": "/opt/web2py/applications/OZtree",
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/python:1": {
"version": "3.10"
}
},
"forwardPorts": [
// This is automatically forwarded if using a local container, but not on Codespaces.
8000
],
"customizations": {
"vscode": {
"extensions": [
"ms-azuretools.vscode-docker"
]
}
}
}
33 changes: 33 additions & 0 deletions .devcontainer/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
version: '3.8'
services:
dev:
build:
context: .
dockerfile: dev.Dockerfile
volumes:
- type: bind
source: ..
target: /opt/web2py/applications/OZtree
consistency: cached
- original-repo:/opt/web2py/applications/OZtree_original:ro
# Overrides default command so things don't shut down after the process ends.
command: /bin/sh -c "while sleep 1000; do :; done"
network_mode: service:web
web:
build:
context: .
dockerfile: web.Dockerfile
args:
IMAGE_NAME: ${WEB_IMAGE_NAME}
volumes:
- type: bind
source: ..
target: /opt/web2py/applications/OZtree
consistency: cached
- original-repo:/opt/web2py/applications/OZtree_original
ports:
- "8080:80"
- "3307:3306"

volumes:
original-repo:
14 changes: 14 additions & 0 deletions .devcontainer/web.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
ARG IMAGE_NAME
FROM ${IMAGE_NAME}
# Based on https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user#_change-the-uidgid-of-an-existing-container-user
# This changes the www-data UID to 1000 to match the default user on the host, so that when it
# chowns everything to www-data, it doesn't make it so that the host user cannot write to the files
# anymore.
RUN groupmod --gid 1000 www-data \
&& usermod --uid 1000 --gid 1000 www-data \
&& chown -R 1000:1000 /opt/web2py/applications
# Move the original source code into a shared volume, clearing out anything from past runs.
RUN if [ -d /opt/web2py/applications/OZtree_original ]; then \
rm -rf /opt/web2py/applications/OZtree_original; \
fi \
&& mv /opt/web2py/applications/OZtree /opt/web2py/applications/OZtree_original
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ OZprivate/rawJS/OZTreeModule/dist/*
OZprivate/rawJS/OZTreeModule/docs/_compiled.markdown
node_modules
OZprivate/ServerScripts/Tests/BlatSearch.code_cache

/static/uikit-3/
.env
/static/uikit-3/
21 changes: 21 additions & 0 deletions README.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,27 @@ If you simply want to run a local copy of OneZoom, but not modify the code yours

# OneZoom setup

## Installing in a Docker Dev Container

If you are using Visual Studio Code or another editor that supports [Dev Containers](https://containers.dev/), the easiest way to set up a full development environment is to use the included Dev Container configuration. This will automatically create two containers: one for development (_dev_), and another (_web_) with the MySQL database and production web server (nginx + uwsgi), derived from the Docker image mentioned above. Your source code will be mounted simultaneously into both containers: dev will let you run build commands and web will serve it as a web server you can access via port forwarding. You can also run your own server from the dev container by [running web2py.py directly](#starting-and-shutting-down-web2py) -- this is particularly useful for debugging.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still trying to grok the model.

and another (web) with the MySQL database

Ok, so only the web container is running mysql. So when running web2py.py directly in the dev container, it's actually connecting to the mysql instance in the web container, thereby going cross container? In other words, they are sharing the DB.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is all correct. I originally tried a one container configuration, extending from the prod Dockerfile. That didn't work; it was missing dev tools and the vscode integration was very buggy. I still wanted to reuse as much of that Dockerfile as possible so I split out the dev container. This is apparently a standard practice when working with databases, e.g. see this template: https://github.com/devcontainers/templates/tree/main/src/postgres

I chose a Node.js base image and added Python as a feature. I could've done the reverse too. Another alternative is we could use the universal Linux image which has both (https://github.com/devcontainers/images/tree/main/src/universal), but we lose control over the Python and Node.js versions (FWIW the current versions match ours). A couple advantages to that are that the builds would be faster (adding a feature like Python is slow) and storage would be free on GitHub Codespaces (this is their default image and they don't charge for storage of containers based on this, except for additional files added). I'm open to that change if folks prefer.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for confirming. I think we should just start with that and see how it goes.

About the DB, I exposed the port in the docker-compose to be able to open it from MySQL workbench (running on Windows). e.g.

    ports:
      - "8080:80"
      - "3307:3306"

I went with 3307 locally to avoid conflicting with my Ubuntu MySQL. Might be worth mentioning that in the doc, and maybe just add that to the default docker-compose.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done and documented.


If you are using Visual Studio Code, perform the following steps (you will need to modify these for another editor):

1. Follow the instructions at either https://hub.docker.com/r/onezoom/oztree or https://hub.docker.com/r/onezoom/oztree-complete to save a docker image with IUCN data.
1. Follow the [VSCode instructions for installing Dev Container support](https://code.visualstudio.com/docs/devcontainers/containers#_installation).
1. `git clone https://github.com/OneZoom/OZtree` into the directory of your choice. If using Windows, it is highly recommended to [clone on the WSL2 filesystem](https://code.visualstudio.com/remote/advancedcontainers/improve-performance#_store-your-source-code-in-the-wsl-2-filesystem-on-windows) both for performance reasons and to avoid permissions issues. If you wish to fork the repository and clone your fork, you will need to copy the tags from upstream, otherwise you will see build issues later. You can do this with `git fetch --tags upstream` followed by `git push --tags`.
Comment thread
wolfmanstout marked this conversation as resolved.
1. Open the cloned directory in VSCode.
1. Create a `.env` file in the `.devcontainer` directory and add `WEB_IMAGE_NAME=onezoom/oztree-with-iucn`, changing the value to whatever image name you choose in step 1.
1. Open the command palette and choose "Dev Containers: Reopen in Container". This may take several minutes to run.
1. Your repository is mounted at `/opt/web2py/applications/OZtree` and the original production docker container application is mounted at `cp /opt/web2py/applications/OZtree_original`. In order to sync your repository with the production database state, open an integrated terminal and run the following: `cp /opt/web2py/applications/OZtree_original/private/appconfig.ini /opt/web2py/applications/OZtree/private/ && cp /opt/web2py/applications/OZtree_original/databases/*.table /opt/web2py/applications/OZtree/databases/`
1. Run `npm ci && grunt dev`. You will need to rerun `grunt dev` any time you make code changes.
1. Visit http://localhost:8080 and the website should load! You can also run your own server from the dev container by [running web2py.py directly](#starting-and-shutting-down-web2py).
1. (Optional) Once tables are created, and everything is working, you can set `migrate = 0` in `private/appconfig.ini`. This will mean that web2py will not make any changes to table structures in the DB, and also that changes to appconfig.ini will require a web2py restart.
1. (Optional) [Create a manager account](#creating-auth-users--groups) in the auth table, e.g. so you can [view docs](#documentation).
1. (Optional) MySQL is available on port 3307 if you wish to debug using local tools outside the container.

## Installing locally

There are two ways in which you can install OneZoom on a personal computer: full installation and partial installation.

* *Partial installation* does not create a standalone OneZoom site, but simply creates a local web file containing the javascript tree viewer. Instead of your tree viewer getting information from your own computer, it must do so by constantly requesting data from the OneZoom website (via the OneZoom APIs). This restricts your OneZoom viewer in various ways: you cannot make your own bespoke tree, you cannot change languages in the viewer, and you are dependent upon a permanent, fast internet connection. Also note that this installation method is also relatively untested, and there are unfixed problems with e.g. displaying lists of popular species. However, partial installation may be suitable for developers who simply want to re-program features of the tree viewer, such as colours, branch geometry, etc.
Expand Down