diff --git a/.github/workflows/ContinuousIntegration.yml b/.github/workflows/ContinuousIntegration.yml index 1b98e636a..399440f70 100644 --- a/.github/workflows/ContinuousIntegration.yml +++ b/.github/workflows/ContinuousIntegration.yml @@ -6,8 +6,18 @@ name: ContinuousIntegration on: push: branches: [ master ] + paths-ignore: + - 'docs/**' + - 'README.md' + - 'CONTRIBUTING.md' + - 'WEBSITE.md' pull_request: branches: [ master ] + paths-ignore: + - 'docs/**' + - 'README.md' + - 'CONTRIBUTING.md' + - 'WEBSITE.md' jobs: ContinuousIntegration: diff --git a/.github/workflows/DocPreviewCleanup.yml b/.github/workflows/DocPreviewCleanup.yml new file mode 100644 index 000000000..e4fac1a97 --- /dev/null +++ b/.github/workflows/DocPreviewCleanup.yml @@ -0,0 +1,36 @@ +# Adapted from +# https://documenter.juliadocs.org/stable/man/hosting/#Cleaning-up-gh-pages + +name: Doc Preview Cleanup + +on: + pull_request: + types: [closed] + +# Ensure that only one "Doc Preview Cleanup" workflow is force pushing at a time +concurrency: + group: doc-preview-cleanup + cancel-in-progress: false + +jobs: + doc-preview-cleanup: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - name: Checkout gh-pages branch + uses: actions/checkout@v4 + with: + ref: gh-pages + - name: Delete preview and history + push changes + run: | + if [ -d "${preview_dir}" ]; then + git config user.name "Documenter.jl" + git config user.email "documenter@juliadocs.github.io" + git rm -rf "${preview_dir}" + git commit -m "delete preview" + git branch gh-pages-new $(echo "delete history" | git commit-tree HEAD^{tree}) + git push --force origin gh-pages-new:gh-pages + fi + env: + preview_dir: docs/previews/PR${{ github.event.number }} diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 41768714f..08222fbfe 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -34,7 +34,6 @@ - In the introduction section(s), provide at least a few sentences of high-level description of the feature, to orient the user. Include external links and internal links. Then, include some short code snippets showing a minimal example of the feature being used. If relevant, include some simple Tex math. -- Build the docs locally to test that it builds and looks right: -```sh -./docs/build_docs_locally.sh -``` +- Build the docs locally to test that it builds and looks right. + +- Once you have a made a PR and are assigned a PR `NUMBER`, update your PR description with a link to the automatically generated documentation preview at https://www.gen.dev/docs/previews/PR{NUMBER}. diff --git a/WEBSITE.md b/WEBSITE.md index ebe6b1a1e..dae407657 100644 --- a/WEBSITE.md +++ b/WEBSITE.md @@ -1,21 +1,24 @@ -The Gen website is hosted on Github pages at https://probcomp.github.io/Gen/ +The Gen website is hosted at https://www.gen.dev/. The website is generated from the source files that are at the head commit of the `gh-pages` branch of the Gen repository. To edit the website, check out the `gh-pages` branch, and commit and push changes. -Automatically-generated Gen documentation is pushed to the `gh-pages` branch whenever a commit is made to the `master` branch of the Gen repository using a combination of Github Actions and Documenter.jl, which are configured in the files `.github/workflows/Documentation.yml` and `docs/make.jl` in the `master` branch. +Automatically-generated Gen documentation is pushed to the `docs` folder of the `gh-pages` branch whenever a commit is made to the `master` branch of the Gen repository using a combination of Github Actions and Documenter.jl, which are configured in the files `.github/workflows/Documentation.yml` and `docs/make.jl` in the `master` branch. When a pull request is merged into the `master` branch, the `gh-pages` branch is updated with the new documentation, and documentation previews will be cleaned up by the `.github/workflows/Documentation.yml` workflow. + The automatically-managed files and directories are: -- `dev/` (documentation for the version of Gen on the head of `master` branch) +- `docs/dev/` (documentation for the version of Gen on the head of `master` branch) + +- `docs/v*.*.*/` (directories that contain documentation for each tagged release of Gen) -- `v*.*.*/` (directories that contain documentation for each tagged release of Gen) +- `docs/latest/` (a symbolic link to `dev/`) -- `latest/` (a symbolic link to `dev/`) +- `docs/stable/` (a symbolic link to the documentation directory of the latest tagged release Gen) -- `stable/` (a symbolic link to the documentation directory of the latest tagged release Gen) +- `docs/previews/` (documentation previews for unmerged pull requests) -- `versions.js` +- `docs/versions.js` *Do not make commits that modify the contents of these automatically managed files and directories.* diff --git a/docs/README.md b/docs/README.md index 78ddd3f69..fcda114d5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,17 +1,18 @@ -# Website Docs -- `pages.jl` to find skeleton of website. -- `make.jl` to build the website index. +# Gen.jl Documentation -The docs are divided in roughly four sections: +- `pages.jl` to find skeleton of the Gen.jl documentation. +- `make.jl` to build the documentation website index. + +The documentation is divided into three sections: - Getting Started + Tutorials - How-to Guides -- API = Modeling API + Inference API -- Explanations + Internals - +- Reference Guides # Developing -To build the docs, run `julia --make.jl` or alternatively startup the Julia REPL and include `make.jl`. For debugging, consider setting `draft=true` in the `makedocs` function found in `make.jl`. -Currently you must write the tutorial directly in the docs rather than a source file (e.g. Quarto). See `getting_started` or `tutorials` for examples. + +To build the docs, run `julia --make.jl`. Alternatively, start the Julia REPL, activate the `Project.toml` in this directory, then include `make.jl`. For debugging, consider setting `draft=true` in the `makedocs` function found in `make.jl`. This will avoid running the `@example` blocks when generating the tutorials. + +Currently you must write the tutorial directly in the docs rather than in a source file. See `tutorials` for examples. Code snippets must use the triple backtick with a label to run. The environment carries over so long as the labels match. Example: @@ -21,4 +22,4 @@ x = rand() ```@example tutorial_1 print(x) -``` \ No newline at end of file +``` diff --git a/docs/make.jl b/docs/make.jl index 92a7cd065..67f66e363 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -8,7 +8,6 @@ makedocs( clean = true, warnonly = true, format = Documenter.HTML(; - assets = String["assets/header.js", "assets/header.css", "assets/theme.css"], collapselevel=1, ), sitename = "Gen.jl", @@ -21,4 +20,5 @@ deploydocs( repo = "github.com/probcomp/Gen.jl.git", target = "build", dirname = "docs", + push_preview = true, ) diff --git a/docs/pages.jl b/docs/pages.jl index dc340eb7e..0f5b89b17 100644 --- a/docs/pages.jl +++ b/docs/pages.jl @@ -1,53 +1,46 @@ pages = [ - "Home" => "index.md", - "Getting Started" => [ - "Example 1: Linear Regression" => "getting_started/linear_regression.md", - ], + "Gen.jl" => "index.md", "Tutorials" => [ - "Basics" => [ - "tutorials/basics/modeling_in_gen.md", - "tutorials/basics/gfi.md", - "tutorials/basics/combinators.md", - "tutorials/basics/particle_filter.md", - "tutorials/basics/vi.md", - ], - "Advanced" => [ - "tutorials/trace_translators.md", - ], - "Model Optmizations" => [ - "Speeding Inference with the Static Modeling Language" => "tutorials/model_optimizations/scaling_with_sml.md", - ], + "Getting Started" => "tutorials/getting_started.md", + "Introduction to Modeling in Gen" => "tutorials/modeling_in_gen.md", + "Object Tracking with SMC" => "tutorials/smc.md", + "Variational Inference in Gen" => "tutorials/vi.md", + "Learning Generative Functions" => "tutorials/learning_gen_fns.md", + "Speeding Up Inference with the SML" => "tutorials/scaling_with_sml.md", ], "How-to Guides" => [ - "MCMC Kernels" => "how_to/mcmc_kernels.md", - "Custom Distributions" => "how_to/custom_distributions.md", - "Custom Modeling Languages" => "how_to/custom_dsl.md", - "Custom Gradients" => "how_to/custom_derivatives.md", - "Incremental Computation" => "how_to/custom_incremental_computation.md", + "Extending Gen" => "how_to/extending_gen.md", + "Adding New Distributions" => "how_to/custom_distributions.md", + "Adding New Generative Functions" => "how_to/custom_gen_fns.md", + "Custom Gradients" => "how_to/custom_gradients.md", + "Custom Incremental Computation" => "how_to/custom_incremental_computation.md", ], - "API Reference" => [ + "Reference" => [ + "Core Interfaces" => [ + "Generative Function Interface" => "ref/core/gfi.md", + "Choice Maps" => "ref/core/choice_maps.md", + "Selections" => "ref/core/selections.md", + "Change Hints" => "ref/core/change_hints.md", + ], "Modeling Library" => [ - "Generative Functions" => "api/model/gfi.md", - "Probability Distributions" => "api/model/distributions.md", - "Choice Maps" => "api/model/choice_maps.md", - "Built-in Modeling Languages" => "api/model/modeling.md", - "Combinators" => "api/model/combinators.md", - "Selections" => "api/model/selections.md", - "Optimizing Trainable Parameters" => "api/model/parameter_optimization.md", - "Trace Translators" => "api/model/trace_translators.md", + "Built-In Modeling Language" => "ref/modeling/dml.md", + "Static Modeling Language" => "ref/modeling/sml.md", + "Probability Distributions" => "ref/modeling/distributions.md", + "Combinators" => "ref/modeling/combinators.md", + "Custom Generative Functions" => "ref/modeling/custom_gen_fns.md", ], "Inference Library" => [ - "Importance Sampling" => "api/inference/importance.md", - "MAP Optimization" => "api/inference/map.md", - "Markov chain Monte Carlo" => "api/inference/mcmc.md", - "MAP Optimization" => "api/inference/map.md", - "Particle Filtering" => "api/inference/pf.md", - "Variational Inference" => "api/inference/vi.md", - "Learning Generative Functions" => "api/inference/learning.md" + "Importance Sampling" => "ref/inference/importance.md", + "Markov Chain Monte Carlo" => "ref/inference/mcmc.md", + "Particle Filtering & SMC" => "ref/inference/pf.md", + "Trace Translators" => "ref/inference/trace_translators.md", + "Parameter Optimization" => "ref/inference/parameter_optimization.md", + "MAP Optimization" => "ref/inference/map.md", + "Variational Inference" => "ref/inference/vi.md", + "Wake-Sleep Learning" => "ref/inference/wake_sleep.md", ], + "Internals" => [ + "Modeling Language Implementation" => "ref/internals/language_implementation.md", + ] ], - "Explanation and Internals" => [ - "Modeling Language Implementation" => "explanations/language_implementation.md", - "explanations/combinator_design.md" - ] ] diff --git a/docs/src/api/inference/map.md b/docs/src/api/inference/map.md deleted file mode 100644 index fcd1ba880..000000000 --- a/docs/src/api/inference/map.md +++ /dev/null @@ -1,4 +0,0 @@ -# MAP Optimization -```@docs -map_optimize -``` diff --git a/docs/src/api/inference/mcmc.md b/docs/src/api/inference/mcmc.md deleted file mode 100644 index bd6df6d00..000000000 --- a/docs/src/api/inference/mcmc.md +++ /dev/null @@ -1,19 +0,0 @@ -# Markov chain Monte Carlo (MCMC) - -Gen supports standard Markov Chain Monte Carlo algorithms and allows users to write their own custom kernels. -```@index -Pages = ["mcmc.md"] -``` - -```@docs -metropolis_hastings -mh -mala -hmc -elliptical_slice -@pkern -@kern -@rkern -reversal -involutive_mcmc -``` diff --git a/docs/src/api/inference/pf.md b/docs/src/api/inference/pf.md deleted file mode 100644 index a356a76d9..000000000 --- a/docs/src/api/inference/pf.md +++ /dev/null @@ -1,10 +0,0 @@ -# Particle Filtering -```@docs -initialize_particle_filter -particle_filter_step! -maybe_resample! -log_ml_estimate -get_traces -get_log_weights -sample_unweighted_traces -``` diff --git a/docs/src/api/inference/vi.md b/docs/src/api/inference/vi.md deleted file mode 100644 index 4d55fb43c..000000000 --- a/docs/src/api/inference/vi.md +++ /dev/null @@ -1,7 +0,0 @@ -## Variational inference -There are two procedures in the inference library for performing black box variational inference. -Each of these procedures can also train the model using stochastic gradient descent, as in a variational autoencoder. -```@docs -black_box_vi! -black_box_vimco! -``` diff --git a/docs/src/api/model/gfi.md b/docs/src/api/model/gfi.md deleted file mode 100644 index 0385988be..000000000 --- a/docs/src/api/model/gfi.md +++ /dev/null @@ -1,55 +0,0 @@ -## [Generative Functions](@id gfi_api) - -```@docs -GenerativeFunction -Trace -``` - -The complete set of methods in the generative function interface (GFI) is: - -```@docs -simulate -generate -update -regenerate -get_args -get_retval -get_choices -get_score -get_gen_fn -Base.getindex -project -propose -assess -has_argument_grads -has_submap -accepts_output_grad -accumulate_param_gradients! -choice_gradients -get_params -``` - -```@docs -Diff -NoChange -UnknownChange -SetDiff -Diffed -``` - -```@docs -CustomUpdateGF -apply_with_state -update_with_state -``` - -```@docs -CustomGradientGF -apply -gradient -``` - -```@docs -Gen.init_update_state -Gen.apply_update! -``` \ No newline at end of file diff --git a/docs/src/api/model/trace_translators.md b/docs/src/api/model/trace_translators.md deleted file mode 100644 index 14de30ec9..000000000 --- a/docs/src/api/model/trace_translators.md +++ /dev/null @@ -1,20 +0,0 @@ -## Trace Translators - -```@docs -@transform -@read -@write -@copy -@tcall -pair_bijections! -is_involution! -inverse -TraceTranslator -DeterministicTraceTranslator -GeneralTraceTranslator -SimpleExtendingTraceTranslator -SymmetricTraceTranslator -``` -```@docs -TraceTransformDSLProgram -``` diff --git a/docs/src/assets/header.css b/docs/src/assets/header.css deleted file mode 100644 index 155b10a3b..000000000 --- a/docs/src/assets/header.css +++ /dev/null @@ -1,138 +0,0 @@ - -@media all and (max-width: 560px) { - header.navigation { - position: fixed !important; - left:0; - top: 0; - width: 100%; - } - - header.navigation div.container { - margin-left: 0rem; - } - - header.navigation div.container nav.navbar { - min-height: 1rem !important; - } - - header.navigation div.container nav.navbar ul.navbar-nav { - min-height: 1rem !important; - margin-left: 0.5rem !important; - } - - header.navigation div.container nav.navbar ul.navbar-nav li.small-item { - visibility: visible !important; - display: block !important; - margin: 0.5rem; - } - - header.navigation div.container nav.navbar ul.navbar-nav li.nav-item { - visibility: hidden; - display: none; - } - - header.navigation div.container nav.navbar ul.navbar-nav li.nav-item a { - visibility: hidden; - display: none; - } - - html:not(.theme--documenter-dark) body #documenter .docs-main { - margin-top: 2rem !important; - } -} - -@media all and (max-width: 1055px) and (min-width: 561px){ - header.navigation { - position: fixed !important; - left:0; - top: 0; - width: 100%; - } - - header.navigation div.container { - margin-left: 0rem; - } - - header.navigation div.container nav.navbar ul.navbar-nav { - width: 80% !important; - } -} - -@media all and (min-width: 1056px) { - header.navigation { - position: fixed !important; - left:0; - top: 0; - width: 100%; - } - - header.navigation div.container { - margin-left: 18rem; - } - -} - -html.theme--documenter-dark header.navigation { - background-color: #1f2424 !important; -} - -html.theme--documenter-dark header.navigation div.container { - border-bottom: 1px solid #5e6d6f; -} - -html.theme--documenter-dark header.navigation div.container nav.navbar { - background-color: #1f2424 !important; -} - -html.theme--documenter-dark header.navigation div.container nav.navbar ul.navbar-nav li.nav-item a.nav-link { - color: white; - transition: color 100ms; -} - -html.theme--documenter-dark header.navigation div.container nav.navbar ul.navbar-nav li.nav-item a.nav-link:hover { - color: #0aa8a7 -} - -html header.navigation { - background-color: white !important; -} - -html header.navigation div.container { - border-bottom: 1px solid #dbdbdb; -} - -html header.navigation div.container nav.navbar ul.navbar-nav li.nav-item a.nav-link { - color: #222; - transition: color 100ms; -} - -html header.navigation div.container nav.navbar ul.navbar-nav li.nav-item a.nav-link:hover { - color: #0aa8a7 -} - -header.navigation { - z-index: 3; -} - -header.navigation div.container nav.navbar ul.navbar-nav { - margin-left: 4rem; - min-height: 3.25rem; - width: 70%; - display: flex; - align-self: auto; - flex-direction: row; - justify-content: space-around; -} - -header.navigation div.container nav.navbar ul.navbar-nav li.nav-item { - align-self: stretch; - align-content: space-around; - justify-content: center; - display: flex; - flex-direction: column; -} - -header.navigation div.container nav.navbar ul.navbar-nav li.small-item { - visibility: hidden; - display: none; -} \ No newline at end of file diff --git a/docs/src/assets/header.js b/docs/src/assets/header.js deleted file mode 100644 index 2f211b385..000000000 --- a/docs/src/assets/header.js +++ /dev/null @@ -1,98 +0,0 @@ -// Source: -// https://github.com/ReactiveBayes/RxInfer.jl/blob/246b196a3ea29d0b5744ce241a923c7a3b30eaf4/docs/src/assets/header.js#L4 - -// We add a simple `onload` hook to inject the custom header for our `HTML`-generated pages -window.onload = function() { - //