Skip to content

Rough Roadmap #6

@mitchmindtree

Description

@mitchmindtree

This is a rough plan for the direction of the repo.

Goals

risc0 rust-toolchain

There are two paths here:

  1. Wrap the binaries similar to rust-overlay.
  2. Compile from scratch.

Path 1: Wrapping Binaries

This approach will be faster to build (as we just fetch binaries), but provides less flexibility around patching behaviour. It'll likely require some serious hacks.

  • Follow the rust-overlay approach of exposing a rust-bin package collection.
    • This is currently implemented, only for the latest 1.85.0 release.
    • This just gets all of the tools running, but there is still runtime behaviour that needs solving.
  • Solve Ideas for getting the rust-toolchain-bin builds working #4 using something like buildFHSUserEnv to wrap the binary with a temporary environment that places the risc0 rust-toolchain under the ~/.rustup directory via symlink. NOTE: This may not be necessary in the future as it looks like newer versions of rzup may accept env vars.
  • Continue from there, patching further as necessary...

Path 2: Compile Components

This approach requires building all the toolchain components from scratch, but provides more flexibility by allowing for patching certain behaviour (e.g. inserting env vars that can be used to override paths, disabling fetching in favour of providing prefetched paths, etc). This would also allow for building the master version of the toolchain without having to wait months for new releases (e.g. risc0's latest released version is 1.85, which is a couple months old).

  • Create a package derivation (i.e. under pkgs/) for each of the rust-toolchain components.
  • Bring all of the components together under one risc0-rust-toolchain derivation using the symlinkJoin nixpkgs builder to mirror the rustup toolchain structure:
    $ tree -L 3 result
    result
    ├── bin
    │   ├── cargo
    │   ├── cargo-clippy
    │   ├── cargo-fmt
    │   ├── clippy-driver
    │   ├── rustc
    │   ├── rustdoc
    │   └── rustfmt
    └── lib
        ├── librustc_driver-7e9901864e0059da.so
        └── rustlib
            ├── riscv32im-risc0-zkvm-elf
            └── x86_64-unknown-linux-gnu
    
  • Patch the source as necessary to override paths/fetching.
  • Setup a binary cache.
    • cachix provides free caches, but have a 5GB limit. Could likely fit the latest couple of versions at least.
    • The magic nix cache seems to be running again, but this isn't accessible outside of the CI environment. We'd likely want something devs can add as a substituter to DL the binaries locally more quickly.

buildRisc0Package, buildRisc0Proof

These can be thought of as small wrappers around stdenv.mkDerivation or buildRustPackage that make any necessary tweaks necessary for buidling risc0 circuits in particular.

Building the guest

The tricky part here will be that technically the guest has a different architecture (riscv32im-risc0-zkvm-elf) from the host, so it will either:

  • Require adding a new target to pkgsCross and using pkgs.pkgsCross.riscv32im-rics0-zkvm.buildRustPackage to cross-compile the guest or
  • Use mkDerivation instead of buildRustPackage and manually build the package using a custom cargo command that specifies the correct target.

Building the proof

For groth16, but trickiest part by far will be fetching and running the docker image in the nix sandbox... at least until there's some native way to build and run it.

It might be possible to do this by implementing a custom docker image "fetcher" (similar to fetchFromGitHub etc, but fetches and hashes the docker image)... It looks like some folks have had mixed success by adding special privileges for the docker daemon? 🥴


Stretch Goals

  • Expose each released semver version of the rust-toolchain under a separate package group (similar to rust-overlay), with a latest shorthand for latest release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions