diff --git a/docs/_static/img/debugging/signature-mismatch1.png b/docs/_static/img/debugging/signature-mismatch1.png new file mode 100644 index 00000000..4e570850 Binary files /dev/null and b/docs/_static/img/debugging/signature-mismatch1.png differ diff --git a/docs/_static/img/debugging/signature-mismatch2.png b/docs/_static/img/debugging/signature-mismatch2.png new file mode 100644 index 00000000..68bd4ea7 Binary files /dev/null and b/docs/_static/img/debugging/signature-mismatch2.png differ diff --git a/docs/_static/img/debugging/signature-mismatch3.png b/docs/_static/img/debugging/signature-mismatch3.png new file mode 100644 index 00000000..c658f7cd Binary files /dev/null and b/docs/_static/img/debugging/signature-mismatch3.png differ diff --git a/docs/_static/img/debugging/signature-mismatch4.png b/docs/_static/img/debugging/signature-mismatch4.png new file mode 100644 index 00000000..12dca53d Binary files /dev/null and b/docs/_static/img/debugging/signature-mismatch4.png differ diff --git a/docs/changelog.md b/docs/changelog.md new file mode 100644 index 00000000..667ccf08 --- /dev/null +++ b/docs/changelog.md @@ -0,0 +1,5 @@ +# Changelog + +```{include} ../CHANGELOG.md +:start-line: 2 +``` diff --git a/docs/conf.py b/docs/conf.py index 6781adc3..9db41351 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -15,6 +15,11 @@ "sphinx.ext.napoleon", "myst_parser", "sphinx_autodoc_typehints", + "sphinx_design", +] + +myst_enable_extensions = [ + "colon_fence", ] intersphinx_mapping = { @@ -38,6 +43,12 @@ html_logo = "_static/img/pyodide-logo.png" html_static_path = ["_static"] +html_theme_options = { + "show_toc_level": 2, + "show_navbar_depth": 2, + "home_page_in_toc": True, +} + sys.path.append(Path(__file__).parent.parent.as_posix()) diff --git a/docs/explanation/architecture.md b/docs/explanation/architecture.md new file mode 100644 index 00000000..7ff29bac --- /dev/null +++ b/docs/explanation/architecture.md @@ -0,0 +1,94 @@ +# How pyodide-build Works + +This page explains the internals of pyodide-build — how it turns a regular Python package into a WebAssembly wheel. + +## The build pipeline + +When you run `pyodide build .`, the following happens: + +``` +┌────────────────┐ +│ pyodide build │ CLI entry point +└───────┬────────┘ + │ + ▼ +┌────────────────┐ +│ Environment │ Install xbuildenv + Emscripten SDK (if needed) +│ setup │ Set up sysconfig, headers, env vars +└───────┬────────┘ + │ + ▼ +┌────────────────┐ +│ pypa/build │ Standard PEP 517 build frontend +│ │ Invokes your build backend (setuptools, meson-python, etc.) +└───────┬────────┘ + │ Build backend calls gcc, g++, cmake, meson, cargo... + ▼ +┌────────────────┐ +│ pywasmcross │ Compiler wrapper — intercepts all tool calls +│ │ Redirects to Emscripten (emcc, em++, emar, etc.) +│ │ Filters incompatible flags, adds Wasm flags +└───────┬────────┘ + │ + ▼ +┌────────────────┐ +│ Emscripten │ Compiles C/C++ → WebAssembly (.o, .a, .so) +│ (emcc/em++) │ Links as SIDE_MODULE +└───────┬────────┘ + │ + ▼ +┌────────────────┐ +│ Wheel output │ .so files (Wasm) + Python files → .whl +│ │ Tagged: pyemscripten_YYYY_P_wasm32 +└────────────────┘ +``` + +## The compiler wrapper (pywasmcross) + +The core of pyodide-build is `pywasmcross` — a compiler wrapper that transparently redirects native compiler calls to Emscripten. + +### How it works + +When pyodide-build sets up the build environment, it creates symlinks named after common compiler tools: + +| Symlink | Redirects to | +|---|---| +| `cc`, `gcc` | `emcc` | +| `c++`, `g++` | `em++` | +| `ar` | `emar` | +| `ranlib` | `emranlib` | +| `strip` | `emstrip` | +| `cmake` | `emcmake cmake` (with toolchain flags) | +| `meson` | `meson` (with cross file injected) | +| `cargo` | `cargo` (with Emscripten target) | + +These symlinks all point to `pywasmcross.py`. When invoked, pywasmcross: + +1. Detects which tool it's impersonating (from the symlink name) +2. Rewrites the command line for Emscripten +3. Filters out incompatible flags +4. Adds WebAssembly-specific flags +5. Executes the real Emscripten tool + +When `pyodide build` runs: + +1. `Makefile.envs` is parsed to set compiler flags, paths, and environment variables +2. The `sysconfig` module is patched to return target-platform values instead of host values +3. `PATH` is modified so pywasmcross symlinks take priority over native compilers +4. Environment variables (`SIDE_MODULE_CFLAGS`, `CMAKE_TOOLCHAIN_FILE`, `MESON_CROSS_FILE`, etc.) are set + +## Build isolation + +By default, `pyodide build` uses [pypa/build](https://build.pypa.io/) in isolated mode — just like `python -m build`. This means: + +1. A temporary virtual environment is created +2. Build dependencies from `[build-system].requires` are installed +3. The build backend is invoked +4. The virtual environment is discarded + +The `--no-isolation` flag skips this and uses the current environment, which is useful for complex builds where you need to manage dependencies yourself. + +## Further reading + +- [Emscripten documentation](https://emscripten.org/docs/) — the underlying compiler toolchain +- [PEP 517](https://peps.python.org/pep-0517/) — Python build system interface diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 00000000..7a60650e --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,47 @@ +# FAQ + +## Which packages work with Pyodide? + +- **Pure Python** — always works. No special tooling needed. +- **C/C++ extensions** — usually works with pyodide-build. Some packages need minor adjustments (e.g., disabling optional native dependencies, guarding platform-specific code). +- **Rust extensions (PyO3)** — works with pyodide-build and the correct Rust nightly toolchain. +- **Threading / multiprocessing** — not supported. Packages that require threads will not work. +- **Networking (sockets)** — not supported. Packages that open raw sockets will not work. High-level HTTP libraries like `requests` and `httpx` have Pyodide-specific fallbacks. +- **Subprocesses** — not supported. Packages that call `subprocess.run()` will not work. + +## Do I need the full Pyodide repository? + +No. pyodide-build is a standalone package. Install it with `pip install pyodide-build` and you're ready to build. You don't need to clone the Pyodide repository. + +## Do I need pyodide-build for pure-Python packages? + +No. A pure-Python wheel built with `python -m build`, `hatch`, `flit`, or any standard build frontend is already compatible with Pyodide. pyodide-build is only needed for packages with compiled extensions (C, C++, Rust). + +## Should I use `pyodide build` directly or cibuildwheel? + +- **cibuildwheel** — if you already build native wheels for Linux/macOS and want to add Pyodide as another platform in the same CI config. +- **`pyodide build` directly** — if you only target Pyodide, want more control, or don't use cibuildwheel yet. + +See [CI with cibuildwheel](how-to/cibuildwheel.md) and [CI without cibuildwheel](how-to/ci-direct.md). + +## Can I use meson-python / scikit-build-core / maturin? + +Yes. pyodide-build supports all major Python build backends: + +- **setuptools** — works out of the box +- **meson-python** — cross file injected automatically. See [Tutorial: Meson](tutorials/meson.md). +- **scikit-build-core** — CMake toolchain handled automatically. See [Tutorial: CMake](tutorials/cmake.md). +- **maturin** — Rust target and flags set automatically. See [Tutorial: Rust](tutorials/rust.md). +- **hatchling / flit** — pure-Python only (no compiled extensions), so no pyodide-build needed. + +## What Node.js version do I need? + +Node.js >= 24 is recommended for `pyodide venv`. Node.js is only needed for testing — not for building. + +## What Python versions are supported? + +pyodide-build requires Python 3.12 or later. The Python version must match the target Pyodide cross-build environment version. + +## What's the difference between `pyodide build` and `python -m build`? + +`pyodide build` wraps `python -m build` with a cross-compilation layer. Your build configuration stays the same — pyodide-build intercepts compiler calls and redirects them to Emscripten. See [Concepts](getting-started/concepts.md) for a detailed comparison. diff --git a/docs/getting-started/concepts.md b/docs/getting-started/concepts.md new file mode 100644 index 00000000..16ef4a91 --- /dev/null +++ b/docs/getting-started/concepts.md @@ -0,0 +1,109 @@ +# Concepts + +This page explains the key ideas behind pyodide-build. Understanding these concepts will help you debug build issues and make sense of the configuration options. + +## Why cross-compilation? + +When you run `python -m build` on your laptop, your C extensions are compiled for your machine's architecture — x86_64 on most desktops, arm64 on Apple Silicon. The resulting wheel only works on that platform. + +Pyodide runs Python inside WebAssembly, which is a completely different compilation target. You can't run `gcc` or `clang` and get a `.so` that works in WebAssembly — you need [Emscripten](https://emscripten.org/), a compiler toolchain that produces WebAssembly output from C/C++ source code. + +pyodide-build automates this cross-compilation. When you run `pyodide build`, it: + +1. Invokes your package's normal build system (setuptools, meson-python, scikit-build-core, etc.) +2. Intercepts all compiler and linker calls +3. Redirects them through Emscripten with the right flags for WebAssembly +4. Produces a standard wheel tagged for the Emscripten platform + +Your build scripts don't need to change — pyodide-build handles the translation transparently. + +## The cross-build environment + +Cross-compilation needs more than just a compiler. Your package's build system needs to find Python headers, link against the right libraries, and query Python's `sysconfig` for the target platform — not the host. The **cross-build environment** (xbuildenv) provides all of this: + +- **CPython headers and sysconfig data** compiled for Emscripten/WebAssembly +- **Pre-built package stubs** for packages like NumPy and SciPy that other packages link against at build time +- **Emscripten SDK** — the compiler toolchain itself (installed automatically) + +When you run `pyodide build`, pyodide-build automatically downloads and sets up the cross-build environment if one isn't already installed. It's cached in your platform's user cache directory so subsequent builds are fast. + +You can also manage the cross-build environment explicitly: + +```bash +pyodide xbuildenv install # install (or update) the cross-build environment +pyodide xbuildenv install 0.27.0 # install a specific Pyodide version +pyodide xbuildenv versions # list installed versions +``` + +See [Managing Cross-Build Environments](../how-to/xbuildenv.md) for more details. + +## Emscripten + +[Emscripten](https://emscripten.org/) is the compiler toolchain that turns C and C++ code into WebAssembly. It provides drop-in replacements for standard compilers: + +| Standard tool | Emscripten equivalent | +|---|---| +| `gcc` / `cc` | `emcc` | +| `g++` / `c++` | `em++` | +| `ar` | `emar` | +| `ranlib` | `emranlib` | + +pyodide-build manages Emscripten automatically — it installs the correct version as part of the cross-build environment and handles all compiler redirection. You don't need to install or configure Emscripten yourself. + +```{important} +Each Pyodide version requires a **specific** Emscripten version. pyodide-build enforces this to ensure ABI compatibility. You can check the required version with `pyodide config get emscripten_version`. +``` + +## Platform tags + +Python wheels include a [platform tag](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/) that identifies which systems they can run on. For example: + +- `manylinux_2_17_x86_64` — Linux on x86_64 +- `macosx_14_0_arm64` — macOS on Apple Silicon +- `pyemscripten_2025_0_wasm32` — Emscripten/WebAssembly + +The Emscripten platform tag, standardized by [PEP 783](https://peps.python.org/pep-0783/), has the format: + +``` +pyemscripten_{year}_{patch}_wasm32 +``` + +Where `{year}_{patch}` is the platform ABI version (e.g., `2025_0`). This version determines which Emscripten SDK version and CPython build are used. Wheels built for one ABI version are **not** compatible with another. + +A complete wheel filename looks like: + +``` +numpy-2.2.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl + │ │ │ │ + │ │ │ └── platform tag + │ │ └── Python ABI tag + │ └── Python version tag + └── package version +``` + +```{note} +Older Pyodide versions (before PEP 783) used the tag `pyodide_{year}_{patch}_wasm32`. The `pyemscripten_*` tag is the standardized form going forward. +``` + +## `pyodide build` vs `python -m build` + +`pyodide build` is designed to be a drop-in replacement for `python -m build` when targeting WebAssembly. Here's what's the same and what's different: + +**The same:** +- Uses your existing `pyproject.toml` build configuration +- Supports the same build backends (setuptools, meson-python, scikit-build-core, hatchling, etc.) +- Produces a standard `.whl` file +- Supports `-C` / `--config-setting` to pass options to the build backend +- Supports `--no-isolation` for custom build environments + +**Different:** +- Compiler calls are intercepted and redirected to Emscripten +- Some compiler flags are filtered out (e.g., `-pthread`, x86 SIMD flags) because they don't apply to WebAssembly +- The output wheel has an Emscripten platform tag instead of a native one +- A cross-build environment must be available (installed automatically on first use) + +## What's next? + +- [Quick Start](quickstart.md) — build your first WebAssembly wheel +- [Managing Cross-Build Environments](../how-to/xbuildenv.md) — advanced xbuildenv management +- [Platform Tags & Compatibility](../reference/platform.md) — full compatibility matrix diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 00000000..036312f2 --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,42 @@ +# Installation + +## Requirements + +- **Linux or macOS** — pyodide-build works best on Linux and macOS. Windows is not supported. +- **Python 3.12 or later** — **must** match the Python version targeted by the Pyodide cross-build environment you install. +- **Node.js** — required for testing with [`pyodide venv`](testing.md). Not needed for building. Node.js >= 24 is recommended. + +## Install pyodide-build + +::::{tab-set} + +:::{tab-item} pip +```bash +pip install pyodide-build +``` +::: + +:::{tab-item} pipx +```bash +pipx install pyodide-build +``` +::: + +:::{tab-item} uv +```bash +uv tool install pyodide-cli --with pyodide-build +``` +::: + +:::: + +Verify the installation: + +```bash +pyodide --version +``` + +## What's next? + +- [Concepts](concepts.md) — understand cross-compilation, the cross-build environment, and platform tags +- [Quick Start](quickstart.md) — build your first WebAssembly wheel diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 00000000..064075bc --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1,72 @@ +# Quick Start + +This guide walks you through building a Python package with C extensions for WebAssembly using pyodide-build. Make sure you've [installed pyodide-build](installation.md) first. + +```{note} +**Pure-Python packages do not need pyodide-build.** If your package has no C, C++, or Rust extensions, a standard wheel built with `python -m build` already works with Pyodide. +``` + +## Build from your source tree + +If you have a package with compiled extensions locally, build it the same way you would with `python -m build`: + +```bash +pyodide build . +``` + +The output wheel is placed in `./dist/` by default: + +``` +dist/your_package-1.0.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl +``` + +On the first run, pyodide-build automatically downloads and sets up the cross-build environment and Emscripten SDK. This may take a minute — subsequent builds are fast. + +You can specify a different output directory with `--outdir` / `-o`: + +```bash +pyodide build . -o wheelhouse/ +``` + +## Passing options to the build backend + +Use `-C` / `--config-setting` to pass options to your build backend, just like with `python -m build`: + +```bash +# Meson project: pass the cross-compilation file +pyodide build . -C setup-args=-Dblas=none -C setup-args=-Dlapack=none + +# setuptools project: pass extra compile args +pyodide build . -C "--build-option=--some-flag" +``` + +## Verify the wheel + +You can inspect the built wheel to confirm it has the correct platform tag: + +```bash +unzip -l dist/your_package-*.whl | head -20 +``` + +The wheel should contain `.so` files (compiled extensions) alongside your Python source, and the filename should include the `pyemscripten_*_wasm32` platform tag. + +## Test the wheel + +Create a [Pyodide virtual environment](testing.md) to test the wheel: + +```bash +pyodide venv .venv-pyodide +source .venv-pyodide/bin/activate +pip install dist/your_package-*.whl +python -c "import your_package; print('it works!')" +``` + +See [Testing with `pyodide venv`](testing.md) for a full walkthrough. + +## What's next? + +- [Testing with `pyodide venv`](testing.md) — verify your wheel in a Pyodide environment +- [CI with cibuildwheel](../how-to/cibuildwheel.md) — automate Pyodide builds in CI +- [Tutorial: Meson Package](../tutorials/meson.md) — building packages with Meson +- [Tutorial: CMake Package](../tutorials/cmake.md) — building packages with CMake +- [Tutorial: Rust Package](../tutorials/rust.md) — building packages with Rust/PyO3 extensions diff --git a/docs/getting-started/testing.md b/docs/getting-started/testing.md new file mode 100644 index 00000000..ece644b6 --- /dev/null +++ b/docs/getting-started/testing.md @@ -0,0 +1,78 @@ +# Testing with `pyodide venv` + +After building a WebAssembly wheel, you'll want to verify it actually works. `pyodide venv` creates a virtual environment where `python` runs on [Pyodide](https://pyodide.org/) via Node.js, so you can test your package in a real WebAssembly runtime without opening a browser. + +## Create a Pyodide virtual environment + +```bash +pyodide venv .venv-pyodide +``` + +This creates a virtual environment at `.venv-pyodide/`. Under the hood, it: + +1. Creates a virtualenv using the Pyodide interpreter (which runs on Node.js) +2. Configures `pip` to only accept WebAssembly-compatible wheels +3. Installs the Pyodide standard library and [micropip](https://micropip.pyodide.org/) + +## Activate and install your wheel + +```bash +source .venv-pyodide/bin/activate +pip install dist/your_package-*.whl +``` + +## Run your code + +Once activated, `python` runs on Pyodide/Node.js: + +```bash +python -c "import your_package; print('it works!')" +``` + +Run your test suite (install pytest first — it's not pre-installed in the venv): + +```bash +pip install pytest +python -m pytest tests/ +``` + +```{important} +Use `python -m pytest` (not bare `pytest`). CLI entry points may not work correctly inside the Pyodide venv — always invoke tools as Python modules. +``` + +## How the venv works + +The Pyodide venv looks like a normal virtualenv, but there are important differences: + +| | Standard venv | Pyodide venv | +|---|---|---| +| `python` | Runs CPython natively | Runs Pyodide on Node.js | +| `pip` | Runs on host Python | Runs on **host Python**, but installs WebAssembly-compatible packages | +| Package compatibility | Any wheel for your platform | Only pure-Python wheels or `pyemscripten_*_wasm32` wheels | + +Key things to know: + +- **`pip` runs on host Python** — it uses your system Python to resolve and download packages, but only installs wheels compatible with WebAssembly. This means `pip install` is fast (no cross-compilation at install time). +- **`python` runs on Pyodide/Node.js** — when you run `python` or `python -c "..."`, it launches Node.js with the Pyodide runtime. This is the real WebAssembly environment. +- **Only binary-compatible wheels are installable** — `pip install` is configured with `only-binary=:all:`, so it won't attempt to build packages from source. If a WebAssembly wheel isn't available, the install will fail. + +## Limitations + +- **Requires Node.js** — the Pyodide venv needs Node.js to run the WebAssembly interpreter. Node.js >= 24 is recommended. +- **No threading** — `threading` and `multiprocessing` are not available in Pyodide. +- **No networking** — `socket`, `http.client`, and similar networking modules don't work. Use `pyodide.http` or `micropip` for HTTP requests. +- **Some tests may need skipping** — tests that rely on threads, subprocesses, networking, or platform-specific behavior will need to be skipped or adapted. Use markers like `@pytest.mark.skipif(sys.platform == "emscripten", ...)`. + +## Recreating the venv + +To recreate the venv from scratch (e.g., after updating pyodide-build): + +```bash +pyodide venv --clear .venv-pyodide +``` + +## What's next? + +- [CI with cibuildwheel](../how-to/cibuildwheel.md) — automate building and testing in CI +- [CI without cibuildwheel](../how-to/ci-direct.md) — set up GitHub Actions directly +- [Debugging Build Failures](../how-to/debugging.md) — troubleshooting when things go wrong diff --git a/docs/how-to/auditwheel.md b/docs/how-to/auditwheel.md new file mode 100644 index 00000000..638a2c44 --- /dev/null +++ b/docs/how-to/auditwheel.md @@ -0,0 +1,90 @@ +# Auditing and Repairing Wheels + +When a Python extension links against shared libraries (`.so` files), those libraries must be available at runtime. On native platforms, `auditwheel` handles this by copying the shared libraries into the wheel. For WebAssembly wheels, pyodide-build uses [auditwheel-emscripten](https://github.com/pyodide/auditwheel-emscripten) via the `pyodide auditwheel` command. + +## Install + +`auditwheel-emscripten` is included as a dependency of pyodide-build and is available as a `pyodide` CLI plugin: + +```bash +pip install pyodide-build +pyodide auditwheel --help +``` + +## When do you need this? + +If your package links against a shared library that isn't included in the wheel after running `python -m build`, +you need to vendor that library into the wheel. Otherwise, the `.so` file will fail to load at runtime with an error like: + +``` +ImportError: unable to load shared library 'libfoo.so' +``` + +## Inspecting a wheel + +Before repairing, inspect the wheel to see what shared libraries it depends on: + +```bash +pyodide auditwheel show dist/your_package-*.whl +``` + +This lists all shared library dependencies of the wheel's `.so` files and shows which are already included and which need to be vendored. + +## Repairing a wheel + +The `repair` command finds the shared library dependencies and copies them into the wheel: + +```bash +pyodide auditwheel repair dist/your_package-*.whl --libdir /path/to/libdir +``` + +The `--libdir` option specifies the directory where the shared libraries are located. + +By default, the wheel is repaired in place, but you can specify an output directory with `-o`: + +```bash +pyodide auditwheel repair dist/your_package-*.whl -o repaired/ +``` + +The repair process: +1. Scans all `.so` files in the wheel for shared library dependencies +2. Locates the required libraries in the specified `--libdir` +3. Copies them into a `.libs/` directory inside the wheel +4. Updates the RPATH of the `.so` files to point to the vendored libraries + +## Typical workflow + +```bash +# 1. Build the wheel +pyodide build . + +# 2. Inspect shared library dependencies +pyodide auditwheel show dist/your_package-*.whl + +# 3. Vendor shared libraries into the wheel +pyodide auditwheel repair dist/your_package-*.whl -w dist/ + +# 4. Test the repaired wheel +pyodide venv .venv-pyodide +source .venv-pyodide/bin/activate +pip install dist/your_package-*.whl +python -m pytest tests/ +``` + +## In cibuildwheel + +Unlike manylinux, cibuildwheel doesn't automatically run auditwheel repair for Pyodide. +This is because we don't know where to locate the shared libraries that the `.so` files depend on. +We cannot find them from the host system paths since we need cross-compiled libraries not the host's libraries. + +Therefore, you need to set the libdir explicitly in the cibuildwheel configuration. + +```toml +[tool.cibuildwheel.pyodide] +repair-wheel-command = "pyodide auditwheel repair --libdir /path/to/libraries --output-dir {dest_dir} {wheel}" +``` + +## What's next? + +- [CI with cibuildwheel](cibuildwheel.md) — cibuildwheel can run auditwheel repair automatically via `repair-wheel-command` +- [Debugging Build Failures](debugging.md) — troubleshooting shared library errors diff --git a/docs/how-to/ci-direct.md b/docs/how-to/ci-direct.md new file mode 100644 index 00000000..a02f5323 --- /dev/null +++ b/docs/how-to/ci-direct.md @@ -0,0 +1,104 @@ +# CI without cibuildwheel + +If you don't use [cibuildwheel](cibuildwheel.md), you can set up GitHub Actions (or any CI) to build Pyodide wheels directly with `pyodide build`. + +## GitHub Actions workflow + +```yaml +# .github/workflows/pyodide.yml +name: Build Pyodide wheel + +on: + push: + tags: ["v*"] + pull_request: + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + + - uses: actions/setup-node@v4 + with: + node-version: "24" + + - name: Install pyodide-build + run: pip install pyodide-build + + - name: Build wheel + run: pyodide build . + + - name: Test wheel + run: | + pyodide venv .venv-pyodide + source .venv-pyodide/bin/activate + pip install dist/*.whl + pip install pytest + python -m pytest tests/ -x + + - uses: actions/upload-artifact@v4 + with: + name: pyodide-wheel + path: dist/*.whl +``` + +## Caching the cross-build environment + +The cross-build environment and Emscripten SDK are downloaded on first use and can be large. Cache them to speed up subsequent runs: + +```yaml + - name: Cache xbuildenv + uses: actions/cache@v4 + with: + path: | + ~/.cache/pyodide + ~/.cache/emsdk + key: pyodide-xbuildenv-${{ hashFiles('**/pyproject.toml') }} + restore-keys: | + pyodide-xbuildenv- +``` + +Place this step before the "Build wheel" step. + +## Passing build options + +Pass options the same way you would locally: + +```yaml + # Meson project + - name: Build wheel + run: pyodide build . -Csetup-args=-Dsome-option=value + + # Custom export mode + - name: Build wheel + run: pyodide build . --exports pyinit +``` + +## Publishing + +Add a publish job that runs on tags: + +```yaml + publish: + needs: build + runs-on: ubuntu-latest + if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v4 + with: + name: pyodide-wheel + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 +``` + +## What's next? + +- [Publishing Wasm Wheels](publishing.md) — more details on distribution +- [CI with cibuildwheel](cibuildwheel.md) — if you also build native wheels diff --git a/docs/how-to/cibuildwheel.md b/docs/how-to/cibuildwheel.md new file mode 100644 index 00000000..88bd774a --- /dev/null +++ b/docs/how-to/cibuildwheel.md @@ -0,0 +1,107 @@ +# CI with cibuildwheel + +[cibuildwheel](https://cibuildwheel.pypa.io/) automates building Python wheels for multiple platforms in CI. Since v2.19.0, it supports Pyodide as a build target — meaning you can build native wheels for Linux, macOS, **and** WebAssembly wheels for Pyodide in the same CI pipeline. + +## Minimal configuration + +Add Pyodide to your `pyproject.toml`: + +```toml +[tool.cibuildwheel] +# Build for CPython 3.13 on all platforms +build = "cp313-*" + +[tool.cibuildwheel.pyodide] +# Test inside a Pyodide venv +test-requires = ["pytest"] +test-command = "python -m pytest {project}/tests -x" +``` + +```{important} +- Use `python -m pytest`, not bare `pytest` — CLI entry points may not work in the Pyodide venv. +``` + +## GitHub Actions workflow + +Pyodide builds must run as a **separate job** with `CIBW_PLATFORM=pyodide` — cibuildwheel won't auto-detect the Pyodide platform. + +```yaml +# .github/workflows/wheels.yml +name: Build wheels + +on: + push: + tags: ["v*"] + pull_request: + +jobs: + build-native: + name: Build wheels on ${{ matrix.os }} + runs-on: ${{ matrix.os }} + strategy: + matrix: + os: [ubuntu-latest, macos-latest] + steps: + - uses: actions/checkout@v4 + - uses: pypa/cibuildwheel@v3.4.0 + - uses: actions/upload-artifact@v4 + with: + name: wheels-${{ matrix.os }} + path: wheelhouse/*.whl + + build-pyodide: + name: Build Pyodide wheels + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: pypa/cibuildwheel@v3.4.0 + env: + CIBW_PLATFORM: pyodide + - uses: actions/upload-artifact@v4 + with: + name: wheels-pyodide + path: wheelhouse/*.whl +``` + +## Key constraints + +| Constraint | Details | +|---|---| +| **Explicit platform** | Must set `CIBW_PLATFORM=pyodide` — auto-detection won't select Pyodide | +| **Wheel repair** | No default repair command — set `repair-wheel-command = ""` if needed | + +## Publishing the wheels + +Combine native and Pyodide wheels in a single publish step: + +```yaml + publish: + needs: [build-native, build-pyodide] + runs-on: ubuntu-latest + if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') + permissions: + id-token: write # trusted publishing + steps: + - uses: actions/download-artifact@v4 + with: + pattern: wheels-* + merge-multiple: true + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 +``` + +## Real-world examples + +These projects use cibuildwheel with Pyodide: + +- **NumPy** — [cibuildwheel config](https://github.com/numpy/numpy/blob/main/pyproject.toml) +- **pandas** — [CI workflow](https://github.com/pandas-dev/pandas/blob/main/.github/workflows/wheels.yml) + +## Further reading + +- [cibuildwheel documentation — Pyodide platform](https://cibuildwheel.pypa.io/en/stable/options/#platform) + +## What's next? + +- [CI without cibuildwheel](ci-direct.md) — set up GitHub Actions with `pyodide build` directly +- [Publishing Wasm Wheels](publishing.md) — distribute your wheels to PyPI diff --git a/docs/how-to/compiler-flags.md b/docs/how-to/compiler-flags.md new file mode 100644 index 00000000..0aa32239 --- /dev/null +++ b/docs/how-to/compiler-flags.md @@ -0,0 +1,88 @@ +# Customizing Compiler Flags + +pyodide-build sets default compiler and linker flags required/optimized for WebAssembly modules. +You can inspect, override, and extend these flags for your build. + +## Default flags + +You can inspect the default values we use by running: + +```bash +pyodide config get cflags +pyodide config get cxxflags +pyodide config get ldflags +``` + +Note that the default flag differs per Pyodide version you target. + +## Overriding flags + +You can override flags in multiple ways: + +### via pyproject.toml + +```toml +[tool.pyodide.build] +cflags = "-O2 -g2" +cxxflags = "-O2 -g2" +ldflags = "-O2 -g2" +``` + +## via environment variables + +Set environment variables before running `pyodide build`: + +```bash +export CFLAGS="$(pyodide config get cflags) -DMY_DEFINE=1" +pyodide build . +``` + +Environment variables take precedence over `pyproject.toml` settings. + +## Configuration precedence + +Flags are resolved in this order (highest priority first): + +1. Environment variables +2. `pyproject.toml` `[tool.pyodide.build]` +3. Cross-build environment defaults + +## Flags that are automatically filtered + +pyodide-build's compiler wrapper automatically removes flags that are incompatible with Emscripten/WebAssembly: + +These are the non-exhaustive list of flags that are filtered. +If you find any flags that are not filtered but should be or vice versa, please let us know. + +| Filtered flag | Reason | +|---|---| +| `-pthread` | Threading is not supported | +| `-bundle`, `-undefined dynamic_lookup` | macOS-specific linker flags | +| `-mpopcnt`, `-mno-sse2`, `-mno-avx2` | x86 SIMD flags (not applicable to Wasm) | +| `-Bsymbolic-functions` | GCC-specific flag not supported by Clang | +| `-fstack-protector` | Not supported in Emscripten | +| `-L/usr/*` | System library paths (not valid for cross-compilation) | + +These are stripped silently — you don't need to remove them from your build scripts. + +## Rust flags + +Rust compiler flags are configured separately: + +```bash +pyodide config get rustflags +# e.g., -C link-arg=-sSIDE_MODULE=2 -C link-arg=-sWASM_BIGINT +``` + +Override in `pyproject.toml`: + +```toml +[tool.pyodide.build] +rustflags = "-C link-arg=-sSIDE_MODULE=2 -C link-arg=-sWASM_BIGINT -C opt-level=2" +``` + +## What's next? + +- [Debugging Build Failures](debugging.md) — troubleshooting when the build fails +- [Configuration Reference](../reference/configuration.md) — all configuration options +- [How pyodide-build Works](../explanation/architecture.md) — how the compiler wrapper works diff --git a/docs/how-to/debugging.md b/docs/how-to/debugging.md new file mode 100644 index 00000000..0e3e0001 --- /dev/null +++ b/docs/how-to/debugging.md @@ -0,0 +1,209 @@ +# Debugging Build Failures + +There are several common failures that can occur when building packages with pyodide-build. This guide helps you identify the phase and apply the right fix. + +## Configuration errors + +### CMake: "Could not find toolchain file" + +The CMake toolchain file wasn't found or passed correctly. + +**Fix**: pyodide-build injects the toolchain automatically. If it doesn't work: + +```bash +CMAKE_TOOLCHAIN_FILE=$(pyodide config get cmake_toolchain_file) +pyodide build . -Ccmake.toolchain="$CMAKE_TOOLCHAIN_FILE" +``` + +### Meson: "Cross file not found" or wrong target architecture + +The Meson cross file wasn't injected or passed correctly. + +**Fix**: + +```bash +MESON_CROSS_FILE=$(pyodide config get meson_cross_file) +pyodide build . -Csetup-args=--cross-file="$MESON_CROSS_FILE" +``` + +## Compilation errors + +### "'something.h' file not found" + +These may fall into two categories: + +1. system headers that is not supported by Emscripten +2. a third party header that is not linked properly + +If the header is from a system library, search it in the Emscripten repository to check +if it's available. If it's not available, you need to update your code to use a different approach. + +If the header is from a third party library, check if it's searched and linked properly. +As you need a cross-compiled version of the library, you need to check if `pkg-config` or `CMake` +or other build systems are configured to find the cross-compiled version correctly. + +### "error: unsupported option '-some-config'" + +It means that the Emscripten compiler doesn't support the option. + +**Fix**: Conditionally disable the flag using the `PYODIDE` environment variable: + +```python +import os +if not os.environ.get("PYODIDE"): + extra_compile_args.append("-some-config") +``` + +### "error: use of undeclared identifier" (platform-specific code) + +Code uses platform-specific APIs (Linux, macOS, Windows) that don't exist in Emscripten. + +**Fix**: Guard with `__EMSCRIPTEN__`: + +```c +#ifdef __EMSCRIPTEN__ +// Emscripten-compatible code +#else +// Platform-specific code +#endif +``` + +## Link errors + +### "wasm-ld: error: undefined symbol: some_function" + +A function is referenced but not defined in any linked object. + +**Fixes**: +- Check if a library needs to be compiled for Emscripten first. +- Make sure you are linking the correct library at the link step. +- Try a different export mode: `pyodide build . --exports whole_archive` +- If the symbol comes from Python itself, it should be available — file an issue + +### "wasm-ld: error: function signature mismatch" + +This is a very common error coming from legacy C/C++ code that was not designed with WebAssembly in mind. + +In C, function signatures are not strictly checked at compile time, but WebAssembly requires strict signature matching. + +``` +wasm-ld: error: function signature mismatch: some_func +>>> defined as (i32, i32) -> i32 in some_static_lib.a(a.o) +>>> defined as (i32) -> i32 in b.o +``` + +**Fix**: Check the signatures of the conflicting functions and fix them to match. + +### "wasm-ld: error: duplicate symbol" + +The same symbol is defined in multiple objects. + +This commonly happens when you are linking the same static library multiple times into different object files. + +**Fixes**: +- Try to switch to the shared library if available. +- Try to avoid linking the same static library multiple times. +- If you are using `--exports whole_archive`, try using `--exports default` instead. It will reduce the number of symbols that are exported. + +### "RuntimeError: function signature mismatch" + +This is a **runtime** error (not a build error) that occurs when calling a function pointer with the wrong type. WebAssembly enforces strict function pointer typing — unlike native platforms which may silently allow mismatches. The most common cause is confusion between `i32` return type and `void` return type in C function pointer casts. + +This is a common but also a tricky error to debug. Here are some steps to help you diagnose and fix it: + +1. Rebuild the package with debug symbols enabled. + +use `CFLAGS='-g2'` compiler flag set to enable debug symbols. This will rename the symbols in the Wasm file to make them more readable. + +2. Open the browser's developer tools and look at the stack trace. + +Use a Chromium-based browser (they have the best Wasm debugging support). +The browser console will show a stack trace — click on the innermost stack frame: + +```{image} /_static/img/debugging/signature-mismatch1.png +:alt: Function signature mismatch stack trace +``` + +Clicking the offset will take you to the corresponding Wasm instruction, which should be a `call_indirect`. This shows the expected function signature: + +```{image} /_static/img/debugging/signature-mismatch2.png +:alt: Wasm call_indirect instruction showing expected signature +``` + +So we think we are calling a function pointer with signature `(param i32 i32) (result i32)` — two `i32` inputs, one `i32` output. Set a breakpoint by clicking on the address, then refresh the page and reproduce the crash. + +Once stopped at the breakpoint, the bottom value on the stack is the function pointer. You can look it up in the console: + +```{image} /_static/img/debugging/signature-mismatch3.png +:alt: Inspecting the function pointer value on the stack +``` + +```javascript +> pyodide._module.wasmTable.get(stack[4].value) // stack[4].value === 13109 +< ƒ $one() { [native code] } +``` + +The bad function pointer's symbol is `one`. Clicking on `$one` brings you to its source, showing the actual signature: + +```{image} /_static/img/debugging/signature-mismatch4.png +:alt: Function pointer actual signature +``` + +The function has signature `(param $var0 i32) (result i32)` — one `i32` input, one `i32` output. This mismatch (called with two args, defined with one) is the cause of the crash. + +3. Find the caller and callee symbols in the source code and fix the type mismatch. If you are not familiar with the codebase, often AI code agents are really good at this kind of task, so delegating this to them can be a good idea. + +## Build system errors + +### pip/build isolation failures + +If the build fails during dependency installation in the isolated environment: + +**Fix**: Use `--no-isolation` and manage dependencies manually: + +```bash +pip install setuptools numpy cython # install build deps +pyodide build . --no-isolation +``` + +## Getting more information + +### Verbose output + +Use `export EMCC_DEBUG=1` to get more detailed output from the Emscripten compiler. + +This can be useful for debugging linker errors and other build issues. + +```bash +export EMCC_DEBUG=1 +pyodide build . +``` + +Increase verbosity to see exactly what commands are being run: + +```bash +# setuptools +pyodide build . -C "--global-option=--verbose" + +# Meson +pyodide build . -Csetup-args=-Dwerror=false -Cbuildtype=debug +``` + +### Check active configuration + +```bash +pyodide config list +``` + +This shows all active build variables including compiler flags, paths, and versions. + +## Useful tools + +- **[Wasm Binary Toolkit (wabt)](https://github.com/WebAssembly/wabt)** — `wasm-objdump`, `wasm2wat`, and other tools for analyzing `.wasm`, `.so`, `.a`, and `.o` files. Essential for diagnosing linker errors and symbol issues. +- **[Emscripten debugging guide](https://emscripten.org/docs/porting/Debugging.html)** — extensive documentation on debugging options available in Emscripten. +- **Chromium DevTools** — use a Chromium-based browser for debugging WebAssembly runtime errors. They have the best Wasm debugging support. + +## What's next? + +- [Customizing Compiler Flags](compiler-flags.md) — adjust flags to fix compilation issues +- [Migrating from Native Builds](migrate.md) — platform differences that cause build failures diff --git a/docs/how-to/migrate.md b/docs/how-to/migrate.md new file mode 100644 index 00000000..f097829c --- /dev/null +++ b/docs/how-to/migrate.md @@ -0,0 +1,136 @@ +# Migrating from Native Builds + +If you already build native wheels with `python -m build` and want to add Pyodide/WebAssembly support, this guide shows what to change and what to watch out for. + +## The short version + +For many packages, the migration is just: + +```bash +pip install pyodide-build +pyodide build . +``` + +Your existing `pyproject.toml`, `setup.py`, `CMakeLists.txt`, or `meson.build` works as-is — pyodide-build handles the cross-compilation transparently. If the build succeeds, you're done. + +This page covers what to do when it doesn't. + +## What works differently in WebAssembly + +WebAssembly (via Emscripten) is a different platform with different capabilities than Linux, macOS, or Windows. Some things your code may rely on are not available: + +### No threading + +`pthread`, `std::thread`, Python's `threading`, and `multiprocessing` are not available. Code that uses them will fail at runtime if not handled properly. + +You need to guard threaded code paths like this: + +```python +import sys + +if sys.platform != "emscripten": + import threading + # threaded implementation +else: + # single-threaded fallback +``` + +### No networking + +`socket`, `http.client`, `urllib.request` (with network I/O), and similar modules don't work. Network access in Pyodide goes through the browser's fetch API. + +**Fix** — guard network code or provide alternative implementations: + +```python +import sys + +if sys.platform == "emscripten": + from pyodide.http import pyfetch + # use pyfetch for HTTP +else: + import urllib.request + # standard networking +``` + + +```{note} +Third party networking libraries such as `requests`, `aiohttp`, `urllib3`, `httpx` +has a Pyodide-specific code path that uses `pyodide.http.pyfetch` or similar alternatives. + +Therefore, if you are using these libraries, your code should work for common use cases. +``` + +### No subprocesses + +`subprocess`, `os.system`, `os.popen`, and related calls don't work in WebAssembly. + +**Fix** — skip or mock subprocess-dependent functionality on Emscripten. + +### 32-bit integers + +WebAssembly is a 32-bit platform. Code that assumes 64-bit pointer sizes or uses pointer-to-int casts may behave differently. + +## Detecting Pyodide at build time + +pyodide-build sets the `PYODIDE` environment variable during the build. Use it to conditionally adjust your build configuration: + +```python +# setup.py or build script +import os + +if os.environ.get("PYODIDE"): + # WebAssembly-specific build adjustments + ... +``` + +For C/C++ code, use the Emscripten preprocessor macro: + +```c +#ifdef __EMSCRIPTEN__ +// WebAssembly-specific code +#else +// Native code +#endif +``` + +## Detecting Pyodide at runtime + +```python +import sys + +if sys.platform == "emscripten": + # Running on Pyodide + ... +``` + +## Common migration patterns + +### Skipping unsupported tests + +Mark tests that require threads, networking, subprocesses, or platform-specific behavior: + +```python +import sys +import pytest + +@pytest.mark.skipif(sys.platform == "emscripten", reason="No threads on Emscripten") +def test_concurrent_access(): + ... + +@pytest.mark.skipif(sys.platform == "emscripten", reason="No sockets on Emscripten") +def test_http_client(): + ... +``` + +## Adding Pyodide to your CI + +Once your build works locally, add it to CI alongside your native builds: + +- [CI with cibuildwheel](cibuildwheel.md) — add Pyodide as a platform in your existing cibuildwheel config +- [CI without cibuildwheel](ci-direct.md) — add a separate GitHub Actions job + +## What's next? + +- [Tutorial: C Extension](../tutorials/c-extension.md) — detailed walkthrough for C extension packages +- [Debugging Build Failures](debugging.md) — systematic troubleshooting +- [Customizing Compiler Flags](compiler-flags.md) — fine-tuning the build diff --git a/docs/how-to/publishing.md b/docs/how-to/publishing.md new file mode 100644 index 00000000..c595703f --- /dev/null +++ b/docs/how-to/publishing.md @@ -0,0 +1,58 @@ +# Publishing Wasm Wheels + +WebAssembly wheels built by pyodide-build are standard Python wheels — they follow the same packaging format and can be published to PyPI like any other wheel. + +```{admonition} PEP 783 +The Emscripten/WebAssembly platform tags are standardized by [PEP 783](https://peps.python.org/pep-0783/), which defines how Python supports the Emscripten platform. This means PyPI and standard Python tooling recognize these wheels as valid platform-specific distributions. +``` + +## Publishing to PyPI + +Upload with [twine](https://twine.readthedocs.io/) or any standard publishing tool: + +```bash +twine upload dist/your_package-*.whl +``` + +Or use [trusted publishing](https://docs.pypi.org/trusted-publishers/) in GitHub Actions (no API tokens needed): + +```yaml + publish: + runs-on: ubuntu-latest + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v4 + with: + name: wheels + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 +``` + +## How users install your package + +Once published, Pyodide users can install your package in two ways: + +### In the browser (via micropip) + +```python +import micropip +await micropip.install("your-package") +``` + +[micropip](https://micropip.pyodide.org/) fetches the wheel from PyPI and installs it in the running Pyodide environment. + +### In a Pyodide venv (via pip) + +```bash +pyodide venv .venv-pyodide +source .venv-pyodide/bin/activate +pip install your-package +``` + +pip resolves the correct `pyemscripten_*_wasm32` wheel automatically. + +## What's next? + +- [CI with cibuildwheel](cibuildwheel.md) — automate building for all platforms +- [CI without cibuildwheel](ci-direct.md) — GitHub Actions with pyodide build directly diff --git a/docs/how-to/xbuildenv.md b/docs/how-to/xbuildenv.md new file mode 100644 index 00000000..664cde1a --- /dev/null +++ b/docs/how-to/xbuildenv.md @@ -0,0 +1,129 @@ +# Managing Cross-Build Environments + +The cross-build environment (xbuildenv) contains everything needed to cross-compile Python packages for WebAssembly: +CPython headers, sysconfig data, and the Emscripten SDK. pyodide-build installs it automatically on first use, but you can also manage it explicitly. + +## Installing + +The `pyodide xbuildenv install` command installs the cross-build environment. + +You *must* have the same host Python version as the one used to build the cross-build environment. +If you have a different Python version, the installation will fail. + +We recommend using `uv` or `pyenv` to manage your Python versions. + +```bash +# Install the latest compatible version +pyodide xbuildenv install + +# Install a specific Pyodide version +pyodide xbuildenv install 0.27.0 + +# Install from a custom URL +pyodide xbuildenv install --url https://example.com/xbuildenv-0.27.0.tar + +# Force install even if version compatibility check fails +pyodide xbuildenv install --force +``` + +## Listing installed versions + +```bash +# List all installed versions (active version marked with *) +pyodide xbuildenv versions +``` + +Output: + +``` +* 0.27.0 + 0.26.4 +``` + +## Switching between versions + +```bash +pyodide xbuildenv use 0.26.4 +``` + +## Checking the current version + +```bash +pyodide xbuildenv version +``` + +## Uninstalling + +```bash +# Uninstall the current version +pyodide xbuildenv uninstall + +# Uninstall a specific version +pyodide xbuildenv uninstall 0.26.4 +``` + +## Searching for available versions + +```bash +# Show versions compatible with your Python and pyodide-build +pyodide xbuildenv search + +# Show all available versions +pyodide xbuildenv search --all + +# Output as JSON (useful for scripting) +pyodide xbuildenv search --json +``` + +The search output shows version compatibility information: + +``` +┌────────────┬────────────┬────────────┬───────────────────────────┬────────────┐ +│ Version │ Python │ Emscripten │ pyodide-build │ Compatible │ +├────────────┼────────────┼────────────┼───────────────────────────┼────────────┤ +│ 0.27.7 │ 3.12.7 │ 3.1.58 │ 0.26.0 - │ Yes │ +│ 0.27.6 │ 3.12.7 │ 3.1.58 │ 0.26.0 - │ Yes │ +└────────────┴────────────┴────────────┴───────────────────────────┴────────────┘ +``` + +## Where the xbuildenv is stored + +pyodide-build resolves the xbuildenv path in this order: + +1. `PYODIDE_XBUILDENV_PATH` environment variable +2. `xbuildenv_path` in `pyproject.toml` under `[tool.pyodide.build]` +3. Platform cache directory (`~/.cache/pyodide` on Linux, `~/Library/Caches/pyodide` on macOS) + +Therefore, to pin a custom location for caching, etc: + +```bash +export PYODIDE_XBUILDENV_PATH=/path/to/xbuildenv +``` + +Or in `pyproject.toml`: + +```toml +[tool.pyodide.build] +xbuildenv_path = "/path/to/xbuildenv" +``` + +## Emscripten SDK + +Each Pyodide version requires a specific Emscripten version. The Emscripten SDK is installed automatically when you run `pyodide build`. +You can also install it manually: + +```bash +pyodide xbuildenv install-emscripten +``` + +Check the required Emscripten version: + +```bash +pyodide config get emscripten_version +``` + +## What's next? + +- [Concepts](../getting-started/concepts.md) — understand what the cross-build environment provides +- [Customizing Compiler Flags](compiler-flags.md) — fine-tuning build flags +- [Configuration Reference](../reference/configuration.md) — all configuration options diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..5d70434e --- /dev/null +++ b/docs/index.md @@ -0,0 +1,125 @@ +# pyodide-build + +**Build Python packages for WebAssembly.** + +pyodide-build is the build toolchain for compiling Python packages to [WebAssembly](https://webassembly.org/) via [Emscripten](https://emscripten.org/). It is the reference implementation of the build toolchain for the [Emscripten/WebAssembly Python platform](https://peps.python.org/pep-0783/). + +This tool is designed to be used with [Pyodide](https://pyodide.org/), but it can also be used with other Emscripten-based Python runtimes that support the same platform tags. + +If you're familiar with `python -m build`, pyodide-build works the same way — just replace it with `pyodide build`: + +```bash +pip install pyodide-build +pyodide build . +``` + +This produces a `.whl` file tagged for the Emscripten platform (e.g., `your_package-1.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl`) that can be published to PyPI and installed in [Pyodide](https://pyodide.org/). + +## Who is this for? + +```{note} +**Pure-Python packages do not need pyodide-build.** A standard wheel built with `python -m build`, `hatch`, `flit`, or any PEP 517 build frontend is already compatible with Pyodide. pyodide-build is for packages that contain compiled extensions. +``` + +pyodide-build is for **Python package maintainers** who want their package to work in WebAssembly environments — the browser, Node.js, or any Emscripten-based Python runtime. Typical users include: + +- **Package authors** adding Pyodide/WebAssembly to their platform support matrix +- **Library maintainers** whose users need the package in [JupyterLite](https://jupyterlite.readthedocs.io/), [Pyodide](https://pyodide.org/), or other browser-based Python environments + +## How it works + +pyodide-build wraps [pypa/build](https://build.pypa.io/) with a cross-compilation layer. When you run `pyodide build`, it: + +1. Sets up a cross-build environment with Emscripten-compiled CPython headers and sysconfig data +2. Intercepts compiler calls (`gcc`, `g++`, `ld`, etc.) and redirects them to Emscripten (`emcc`, `em++`) +3. Translates compiler flags for WebAssembly compatibility +4. Produces a standard wheel with the appropriate platform tag + +Your existing `setup.py`, `pyproject.toml`, CMakeLists.txt, or `meson.build` works as-is — pyodide-build handles the cross-compilation transparently. + +## Where to start + +::::{grid} 1 1 2 3 +:gutter: 2 + +:::{grid-item-card} Quick Start +:link: getting-started/quickstart +Build your first WebAssembly wheel in 5 minutes. +::: + +:::{grid-item-card} CI with cibuildwheel +:link: how-to/cibuildwheel +Add Pyodide to your existing cibuildwheel CI pipeline. +::: + +:::{grid-item-card} Testing with `pyodide venv` +:link: getting-started/testing +Verify your wheel works in a Pyodide environment. +::: + +:::: + +```{toctree} +:maxdepth: 2 +:hidden: +:caption: Getting Started + +getting-started/installation +getting-started/concepts +getting-started/quickstart +getting-started/testing +``` + +```{toctree} +:maxdepth: 2 +:hidden: +:caption: Tutorials + +tutorials/c-extension +tutorials/meson +tutorials/cmake +tutorials/rust +``` + +```{toctree} +:maxdepth: 2 +:hidden: +:caption: How-to Guides + +how-to/cibuildwheel +how-to/ci-direct +how-to/publishing +how-to/migrate +how-to/xbuildenv +how-to/compiler-flags +how-to/auditwheel +how-to/debugging +``` + +```{toctree} +:maxdepth: 2 +:hidden: +:caption: Reference + +reference/cli +reference/configuration +reference/platform +explanation/architecture +``` + +```{toctree} +:maxdepth: 2 +:hidden: +:caption: Appendix + +faq +changelog +``` + +## Communication + +- Discord: [Pyodide Discord](https://dsc.gg/pyodide) +- Blog: [blog.pyodide.org](https://blog.pyodide.org/) +- Mailing list: [mail.python.org/mailman3/lists/pyodide.python.org/](https://mail.python.org/mailman3/lists/pyodide.python.org/) +- X: [x.com/pyodide](https://x.com/pyodide) +- Stack Overflow: [stackoverflow.com/questions/tagged/pyodide](https://stackoverflow.com/questions/tagged/pyodide) diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index 28b521a4..00000000 --- a/docs/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -pyodide-build -============= - -``pyodide-build`` is the build toolchain for `PEP 783 `_. - -.. toctree:: - :maxdepth: 2 - :caption: Contents: - - -Communication -------------- - -- Blog: `blog.pyodide.org `_ -- Mailing list: `mail.python.org/mailman3/lists/pyodide.python.org/ `_ -- Gitter: `gitter.im/pyodide/community `_ -- Twitter: `twitter.com/pyodide `_ -- Stack Overflow: `stackoverflow.com/questions/tagged/pyodide `_ diff --git a/docs/reference/cli.md b/docs/reference/cli.md new file mode 100644 index 00000000..7a955adc --- /dev/null +++ b/docs/reference/cli.md @@ -0,0 +1,149 @@ +# CLI Reference + +All commands are accessed through the `pyodide` CLI, provided by the [pyodide-cli](https://pypi.org/project/pyodide-cli/) package (installed automatically with pyodide-build). + +## pyodide build + +Build a Python package for WebAssembly. + +``` +pyodide build [OPTIONS] [SOURCE_LOCATION] +``` + +**Arguments:** + +| Argument | Description | +|---|---| +| `SOURCE_LOCATION` | Build source: a directory. Defaults to the current directory. | + +**Options:** + +| Option | Default | Description | +|---|---|---| +| `-o`, `--outdir` | `./dist` | Output directory for the built wheel | +| `--exports` | `requested` | Symbol export mode: `pyinit`, `requested`, `whole_archive`, or comma-separated list | +| `-C`, `--config-setting` | | Pass settings to the build backend (same as `pypa/build`) | +| `-n`, `--no-isolation` | `false` | Disable build isolation; build deps must be installed manually | +| `-x`, `--skip-dependency-check` | `false` | Skip build dependency check (only with `--no-isolation`) | +| `--compression-level` | `6` | Zip compression level for the wheel | +| `--xbuildenv-path` | platform cache | Path to the cross-build environment, inferred from platform cache if not specified | + +## pyodide venv + +Create a Pyodide virtual environment for testing. + +``` +pyodide venv [OPTIONS] DEST +``` + +**Arguments:** + +| Argument | Description | +|---|---| +| `DEST` | Directory to create the virtualenv at | + +**Options:** + +| Option | Default | Description | +|---|---|---| +| `--clear` / `--no-clear` | `no-clear` | Remove destination directory if it exists | +| `--no-vcs-ignore` | | Don't create VCS ignore directive (e.g., `.gitignore`) | +| `--download` / `--no-download` | | Enable/disable download of latest pip/setuptools from PyPI | +| `--extra-search-dir` | | Path containing additional wheels | +| `--pip` | `bundle` | pip version: `embed`, `bundle`, or exact version | +| `--setuptools` | | setuptools version: `embed`, `bundle`, `none`, or exact version | +| `--no-setuptools` | | Do not install setuptools | + +## pyodide config + +Query build configuration values. + +### pyodide config list + +``` +pyodide config list +``` + +Lists all config variables and their current values. + +### pyodide config get + +``` +pyodide config get CONFIG_VAR +``` + +Get a single config variable's value. Common variables: + +| Variable | Description | +|---|---| +| `cflags` | C compiler flags | +| `cxxflags` | C++ compiler flags | +| `ldflags` | Linker flags | +| `rustflags` | Rust compiler flags | +| `cmake_toolchain_file` | Path to CMake toolchain file | +| `meson_cross_file` | Path to Meson cross file | +| `rust_toolchain` | Required Rust nightly toolchain | +| `emscripten_version` | Required Emscripten version | +| `python_version` | Target Python version | +| `xbuildenv_path` | Cross-build environment path | +| `pyodide_abi_version` | Pyodide ABI version | + +## pyodide xbuildenv + +Manage the cross-build environment. + +### pyodide xbuildenv install + +``` +pyodide xbuildenv install [OPTIONS] [VERSION] +``` + +| Option | Env var | Description | +|---|---|---| +| `--path` | `PYODIDE_XBUILDENV_PATH` | Destination directory | +| `--url` | | Download from a custom URL | +| `-f`, `--force` | | Force install even if version is incompatible | + +### pyodide xbuildenv version + +``` +pyodide xbuildenv version [--path PATH] +``` + +Print the current active version. + +### pyodide xbuildenv versions + +``` +pyodide xbuildenv versions [--path PATH] +``` + +List all installed versions. Active version is marked with `*`. + +### pyodide xbuildenv use + +``` +pyodide xbuildenv use VERSION [--path PATH] +``` + +Switch to a specific installed version. + +### pyodide xbuildenv uninstall + +``` +pyodide xbuildenv uninstall [VERSION] [--path PATH] +``` + +Uninstall a version. Defaults to the current version if not specified. + +### pyodide xbuildenv search + +``` +pyodide xbuildenv search [OPTIONS] +``` + +| Option | Description | +|---|---| +| `--metadata` | Custom metadata file URL or path | +| `-a`, `--all` | Show all versions, including incompatible ones | +| `--json` | Output as JSON | diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 00000000..90fff9fc --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,57 @@ +# Configuration Reference + +pyodide-build reads configuration from multiple sources. This page documents all available settings. + +## Precedence + +Configuration is resolved in this order (highest priority first): + +1. **Environment variables** +2. **`pyproject.toml`** under `[tool.pyodide.build]` +3. **Cross-build environment** defaults +4. **Built-in defaults** + +## pyproject.toml + +Add configuration under `[tool.pyodide.build]`: + +```toml +[tool.pyodide.build] +cflags = "-O2" +cxxflags = "-O2" +ldflags = "-s SIDE_MODULE=1 -O2" +xbuildenv_path = "/path/to/xbuildenv" +``` + +## User-overridable settings + +These keys can be set in `pyproject.toml` or via environment variables: + +| pyproject.toml key | Env variable | Description | +|---|---|---| +| `cflags` | `CFLAGS` | C compiler flags | +| `cxxflags` | `CXXFLAGS` | C++ compiler flags | +| `ldflags` | `LDFLAGS` | Linker flags | +| `rustflags` | `RUSTFLAGS` | Rust compiler flags | +| `rust_toolchain` | `RUST_TOOLCHAIN` | Rust nightly toolchain version | +| `meson_cross_file` | `MESON_CROSS_FILE` | Path to Meson cross file | +| `xbuildenv_path` | `PYODIDE_XBUILDENV_PATH` | Path to cross-build environment | +| `ignored_build_requirements` | `IGNORED_BUILD_REQUIREMENTS` | Build requirements to ignore | + +## Environment variables + +### Querying configuration + +Use `pyodide config` to inspect active values: + +```bash +# List all settings +pyodide config list + +# Get a specific value +pyodide config get cflags +pyodide config get meson_cross_file +pyodide config get rust_toolchain +``` + +See the [CLI Reference](cli.md) for details. diff --git a/docs/reference/platform.md b/docs/reference/platform.md new file mode 100644 index 00000000..f1ad0425 --- /dev/null +++ b/docs/reference/platform.md @@ -0,0 +1,62 @@ +# Platform Tags & Compatibility + +The Emscripten/WebAssembly platform for Python is standardized by [PEP 783](https://peps.python.org/pep-0783/). + +## Platform tag format + +Wheels built by pyodide-build use the platform tag: + +``` +pyemscripten_{year}_{patch}_wasm32 +``` + +A complete wheel filename: + +``` +numpy-2.2.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl + │ │ │ │ + │ │ │ └── platform tag + │ │ └── Python ABI tag + │ └── Python version tag + └── package version +``` + +## Compatibility matrix + +Each platform version is tied to a specific Python version and Emscripten SDK version. Wheels built for one platform version are **not** compatible with another. + +| Platform tag | Python | Emscripten | Notes | +|---|---|---|---| +| `pyodide_2024_0_wasm32` | 3.12 | 3.1.58 | Legacy tag name | +| `pyemscripten_2025_0_wasm32` | 3.13 | 4.0.9 | PEP 783 standardized name | +| `pyemscripten_2026_0_wasm32` | 3.14 | TBD | Under development | + +```{note} +Older Pyodide versions used the tag `pyodide_{year}_{patch}_wasm32`. The `pyemscripten_*` tag is the standardized form going forward per PEP 783. +``` + +## ABI compatibility rules + +- Wheels are **not cross-version compatible** — a wheel built for `pyemscripten_2025_0_wasm32` will not work with `pyemscripten_2024_0_wasm32` or `pyemscripten_2026_0_wasm32`. +- Pure-Python wheels (`py3-none-any`) work on all versions. +- The ABI version determines which Emscripten SDK and CPython build are used. Mixing versions will cause load-time or runtime errors. + +## cibuildwheel identifiers + +When using [cibuildwheel](../how-to/cibuildwheel.md), the platform identifiers are: + +| cibuildwheel identifier | Platform tag | +|---|---| +| `cp312-pyodide_wasm32` | `pyodide_2024_0_wasm32` | +| `cp313-pyodide_wasm32` | `pyemscripten_2025_0_wasm32` | + +## Checking your platform version + +```bash +pyodide config get pyodide_abi_version +``` + +## Further reading + +- [PEP 783 — The Emscripten Platform](https://peps.python.org/pep-0783/) — formal specification +- [Concepts — Platform Tags](../getting-started/concepts.md) — introductory explanation diff --git a/docs/requirements-doc.txt b/docs/requirements-doc.txt index 63d8a5d2..0ae1b9f6 100644 --- a/docs/requirements-doc.txt +++ b/docs/requirements-doc.txt @@ -2,3 +2,4 @@ sphinx>=5.3.0 sphinx_book_theme>=0.4.0rc1 myst-parser sphinx-autodoc-typehints>=1.21.7 +sphinx-design diff --git a/docs/tutorials/c-extension.md b/docs/tutorials/c-extension.md new file mode 100644 index 00000000..c8c54731 --- /dev/null +++ b/docs/tutorials/c-extension.md @@ -0,0 +1,216 @@ +# Tutorial: Package with C Extension + +This tutorial walks through building a Python package with a C extension for WebAssembly. We'll start with a minimal example, build it, test it, and then cover what to do when things go wrong. + +## Example package + +Consider a package `fastcount` with this layout: + +``` +fastcount/ +├── pyproject.toml +├── fastcount/ +│ ├── __init__.py +│ └── _core.c +``` + +`pyproject.toml`: + +```toml +[build-system] +requires = ["setuptools"] +build-backend = "setuptools.build_meta" + +[project] +name = "fastcount" +version = "1.0.0" + +[tool.setuptools] +ext-modules = [ + {name = "fastcount._core", sources = ["fastcount/_core.c"]} +] +``` + +`fastcount/__init__.py`: + +```python +from fastcount._core import count_chars +``` + +`fastcount/_core.c`: + +```c +#define PY_SSIZE_T_CLEAN +#include + +static PyObject* count_chars(PyObject* self, PyObject* args) { + const char* str; + char target; + if (!PyArg_ParseTuple(args, "sC", &str, &target)) + return NULL; + + long count = 0; + for (const char* p = str; *p; p++) { + if (*p == target) count++; + } + return PyLong_FromLong(count); +} + +static PyMethodDef methods[] = { + {"count_chars", count_chars, METH_VARARGS, "Count occurrences of a character."}, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef module = { + PyModuleDef_HEAD_INIT, "_core", NULL, -1, methods +}; + +PyMODINIT_FUNC PyInit__core(void) { + return PyModule_Create(&module); +} +``` + +## Build it + +```bash +pyodide build . +``` + +That's it. pyodide-build: + +1. Invokes setuptools to compile `_core.c` +2. Intercepts the `gcc`/`cc` call and redirects it to Emscripten's `emcc` +3. Links the compiled WebAssembly into a `.so` file (which is actually a Wasm binary) +4. Packages everything into a wheel with the `pyemscripten_*_wasm32` platform tag + +Output: + +``` +dist/fastcount-1.0.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl +``` + +## Test it + +```bash +pyodide venv .venv-pyodide +source .venv-pyodide/bin/activate +pip install dist/fastcount-*.whl +python -c "from fastcount import count_chars; print(count_chars('hello world', 'l'))" +# Output: 3 +``` + +## What happens under the hood + +When pyodide-build intercepts a compiler call, it does more than just swap `gcc` for `emcc`. It also: + +- **Filters incompatible flags** — flags like `-pthread`, `-mpopcnt`, `-mno-sse2`, and macOS-specific flags (`-bundle`, `-undefined dynamic_lookup`) are silently removed because they don't apply to WebAssembly. +- **Adds Emscripten flags** — flags like `-s SIDE_MODULE=1` are added to produce a loadable WebAssembly module. + +You can see exactly what compiler commands are being run by setting `EMCC_DEBUG=1`: + +```bash +EMCC_DEBUG=1 pyodide build . +``` + +## Export modes + +When linking a `.so` file for WebAssembly, pyodide-build needs to know which symbols to export (make visible to the Python runtime). The `--exports` flag controls this: + +| Mode | What it exports | When to use | +|---|---|---| +| `pyinit` | Only `PyInit_*` functions | Minimal exports. Works for standard Python C extensions that don't need to share symbols. | +| `requested` | All public symbols from object files | **Default.** Use when other extensions or packages may need to link against your symbols at runtime. | +| `whole_archive` | Everything | No filtering at all. Use for shared libraries or when the other modes cause missing symbol errors. Produces larger files. | + +The default is `requested`. For most packages, you don't need to change this: + +```bash +# Only if you need a different export mode: +pyodide build . --exports pyinit +pyodide build . --exports whole_archive +``` + +You can also export specific symbols by name: + +```bash +pyodide build . --exports "my_func1,my_func2" +``` + +## Common build issues + +### Missing header files + +``` +fatal error: 'some_library.h' file not found +``` + +This means your C code includes a header that isn't available in the Emscripten. + +If it's a system library, it needs to be cross-compiled for Emscripten first. pyodide-build does not allow reading headers from the host system's include paths. +This is because linking the library built for the host system to the WebAssembly module will cause build time / runtime errors. + +### Undefined symbols at link time + +``` +wasm-ld: error: undefined symbol: some_function +``` + +This usually means your extension depends on a C library that hasn't been compiled for WebAssembly. Check: +- Is the library available in the Emscripten sysroot? +- Does the library need to be cross-compiled for WebAssembly first? +- Try `--exports whole_archive` if the symbol should be coming from your own code. + +### Unsupported compiler features + +``` +error: unsupported option '-pthread' +``` + +pyodide-build filters out most incompatible flags automatically, but some may slip through. +You may need to conditionally disable them: + +```python +# setup.py +import os + +extra_compile_args = ["-O2"] +if os.environ.get("PYODIDE"): + # Skip flags that don't work in WebAssembly + pass +else: + extra_compile_args.append("-pthread") +``` + +The `PYODIDE` environment variable is set by pyodide-build during the build process. + +### Function pointer type mismatch + +``` +RuntimeError: function signature mismatch +``` + +WebAssembly enforces strict function pointer typing. If your C code casts function pointers to incompatible types (common in older C code), you'll get this error at runtime, not at build time. The fix is to ensure function pointer types match exactly. + +```{Tip} +This error is very common but often very tricky to debug by human eyes. Usually LLMs are quite helpful in identifying and fixing this issue. +``` + +## Using Cython + +Cython extensions work the same way — pyodide-build intercepts the C compilation step that Cython generates: + +```toml +[build-system] +requires = ["setuptools", "cython"] +build-backend = "setuptools.build_meta" +``` + +No special configuration is needed for Cython. Just run `pyodide build .` as usual. + +## What's next? + +- [Tutorial: Meson Package](meson.md) — building packages with Meson +- [Tutorial: CMake Package](cmake.md) — building packages with CMake +- [Tutorial: Rust Package](rust.md) — building packages with PyO3/Maturin +- [Customizing Compiler Flags](../how-to/compiler-flags.md) — fine-tuning the build +- [Debugging Build Failures](../how-to/debugging.md) — systematic troubleshooting diff --git a/docs/tutorials/cmake.md b/docs/tutorials/cmake.md new file mode 100644 index 00000000..e8f89255 --- /dev/null +++ b/docs/tutorials/cmake.md @@ -0,0 +1,41 @@ +# Tutorial: CMake Package + +Python packages that use [CMake](https://cmake.org/) — typically via [scikit-build-core](https://scikit-build-core.readthedocs.io/) — are supported by pyodide-build. + +## Basic usage + +If your package uses scikit-build-core or another build system that invokes CMake, you can build it directly with pyodide-build: + +```bash +pyodide build . +``` + +pyodide-build automatically intercepts `cmake` calls and configures the Emscripten toolchain, including compiler paths and build flags. + +## What the toolchain provides + +pyodide-build has a custom CMake toolchain that is used to configure the build to target Emscripten. + +The CMake toolchain that pyodide-build injects: + +- Inherits from Emscripten's own CMake toolchain +- It sets the compiler and linkers +- Sets up library search paths for the cross-build environment + +```{tip} +If automatic toolchain injection doesn't work for your setup, you can pass it explicitly: + + CMAKE_TOOLCHAIN_FILE=$(pyodide config get cmake_toolchain_file) + pyodide build . -Ccmake.toolchain="$CMAKE_TOOLCHAIN_FILE" + +Or via the `CMAKE_ARGS` environment variable: + + export CMAKE_ARGS="-DCMAKE_TOOLCHAIN_FILE=$(pyodide config get cmake_toolchain_file)" + pyodide build . +``` + +## What's next? + +- [Tutorial: Meson Package](meson.md) — building packages with Meson / meson-python +- [Tutorial: Rust Package](rust.md) — building packages with PyO3/Maturin +- [Customizing Compiler Flags](../how-to/compiler-flags.md) — fine-tuning compiler and linker flags diff --git a/docs/tutorials/meson.md b/docs/tutorials/meson.md new file mode 100644 index 00000000..1a63eed1 --- /dev/null +++ b/docs/tutorials/meson.md @@ -0,0 +1,61 @@ +# Tutorial: Meson Package + +Many scientific Python packages use [Meson](https://mesonbuild.com/) via [meson-python](https://meson-python.readthedocs.io/) as their build system. pyodide-build supports Meson out of the box. + +## Basic usage + +For most Meson packages, just run: + +```bash +pyodide build . +``` + +pyodide-build automatically intercepts `meson setup` calls and injects the Meson cross file, which tells Meson the build is targeting Emscripten. No manual configuration is needed. + +## Passing Meson options + +Use `-Csetup-args=` to pass options to Meson: + +```bash +pyodide build . \ + -Csetup-args=-Dsome-option=value \ + -Csetup-args=-Danother-option=false +``` + +Each Meson option needs its own `-Csetup-args=` prefix. + +## Real-world example: NumPy + +NumPy uses meson-python. Here's how to build it for WebAssembly: + +```bash +pyodide build . -Csetup-args=-Dallow-noblas=true +``` + +The `-Dallow-noblas=true` disables BLAS/LAPACK (which aren't available as Emscripten libraries by default). This pattern — disabling optional native dependencies — is common when cross-compiling. + +## What the cross file provides + +The Meson cross file that pyodide-build injects tells Meson: + +- The host machine is `emscripten` / `wasm32` +- Use `node` as the executable wrapper +- Skip the compiler sanity check (Emscripten's output can't run natively) + +```{tip} +If automatic cross file injection doesn't work for your setup, you can pass it explicitly: + + MESON_CROSS_FILE=$(pyodide config get meson_cross_file) + pyodide build . -Csetup-args=--cross-file="$MESON_CROSS_FILE" + +Or via the `MESON_CROSS_FILE` environment variable: + + export MESON_CROSS_FILE=$(pyodide config get meson_cross_file) + pyodide build . +``` + +## What's next? + +- [Tutorial: CMake Package](cmake.md) — building packages with CMake / scikit-build-core +- [Tutorial: Rust Package](rust.md) — building packages with PyO3/Maturin +- [Customizing Compiler Flags](../how-to/compiler-flags.md) — fine-tuning compiler and linker flags diff --git a/docs/tutorials/rust.md b/docs/tutorials/rust.md new file mode 100644 index 00000000..7ba2dfbf --- /dev/null +++ b/docs/tutorials/rust.md @@ -0,0 +1,81 @@ +# Tutorial: Rust Package + +Python packages with Rust extensions — typically built with [PyO3](https://pyo3.rs/) and [Maturin](https://www.maturin.rs/) — can be cross-compiled for WebAssembly using pyodide-build. + +```{note} +Rust support in pyodide-build requires some manual setup compared to C/C++ packages. You need to install the correct Rust toolchain and Emscripten target yourself. +``` + +## Prerequisites + +### 1. Install Rust + +If you don't have Rust installed, use [rustup](https://rustup.rs/): + +```bash +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +``` + +### 2. Install the correct Rust toolchain + +pyodide-build requires a specific minimum Rust toolchain version. Query it with: + +```bash +pyodide config get rust_toolchain +``` + +Install and activate it: + +```bash +RUST_TOOLCHAIN=$(pyodide config get rust_toolchain) +rustup toolchain install "$RUST_TOOLCHAIN" +rustup default "$RUST_TOOLCHAIN" +``` + +### 3. Add the Emscripten target + +The `wasm32-unknown-emscripten` target must be available for your Rust toolchain: + +```bash +RUST_TOOLCHAIN=$(pyodide config get rust_toolchain) +rustup target add wasm32-unknown-emscripten --toolchain "$RUST_TOOLCHAIN" +``` + +## Build a Rust package + +With the toolchain set up, build your Maturin/PyO3 package: + +```bash +pyodide build . +``` + +pyodide-build sets the following environment variables automatically during the build: + +| Variable | Value | Purpose | +|---|---|---| +| `CARGO_BUILD_TARGET` | `wasm32-unknown-emscripten` | Tells Cargo to cross-compile for Emscripten | +| `CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_LINKER` | `emcc` | Uses Emscripten as the linker | +| `PYO3_CROSS_INCLUDE_DIR` | `/include` | PyO3 cross-compilation: Python headers location | +| `PYO3_CROSS_LIB_DIR` | `/lib` | PyO3 cross-compilation: Python library location | +| `PYO3_CROSS_PYTHON_VERSION` | `3.x` | PyO3 cross-compilation: target Python version | + +You don't need to set these manually — they come from the cross-build environment. + +## Querying Rust configuration + +Use `pyodide config` to inspect the Rust-related build settings: + +```bash +pyodide config get rust_toolchain # e.g., nightly-2025-02-01 +pyodide config get rustflags # e.g., -C link-arg=-sSIDE_MODULE=2 ... +``` + +## Limitations + +- **Nightly Rust required** — Before Python 3.14, the `wasm32-unknown-emscripten` target is not available on stable Rust. The specific nightly version must match what pyodide-build expects. + +## What's next? + +- [Tutorial: C Extension](c-extension.md) — building packages with C extensions +- [Customizing Compiler Flags](../how-to/compiler-flags.md) — fine-tuning compiler and linker flags +- [Configuration Reference](../reference/configuration.md) — all `pyodide config` values