diff --git a/src/docs/index.rst b/src/docs/index.rst index f1c192a414..669d355282 100644 --- a/src/docs/index.rst +++ b/src/docs/index.rst @@ -3,9 +3,9 @@ .. ## .. ## SPDX-License-Identifier: (BSD-3-Clause) -======= +===== Smith -======= +===== Smith is a 3D implicit nonlinear thermal-structural simulation code. Its primary purpose is to investigate multiphysics abstraction strategies and implicit finite element-based algorithm development for emerging @@ -15,13 +15,12 @@ computing architectures. It heavily leverages the `MFEM finite element library < This project is under heavy development and is currently a pre-alpha release. Functionality and interfaces may change rapidly as development progresses. -* :ref:`Quickstart/Build Instructions ` -* `Source Documentation `_ + +We also provide `Source API Documentation `_. -====================================================== Copyright and License Information -====================================================== +--------------------------------- Please see the `LICENSE `_ file in the repository. @@ -30,11 +29,17 @@ Produced at the Lawrence Livermore National Laboratory. LLNL-CODE-805541 +.. toctree:: + :hidden: + :titlesonly: + :maxdepth: 2 + + sphinx/build_guide/index + .. toctree:: :hidden: :maxdepth: 2 - sphinx/quickstart sphinx/user_guide/index sphinx/dev_guide/index sphinx/theory_reference/index diff --git a/src/docs/requirements.txt b/src/docs/requirements.txt index 0a42ee80cd..121ba16c39 100644 --- a/src/docs/requirements.txt +++ b/src/docs/requirements.txt @@ -1,3 +1,3 @@ docutils -sphinx==6.2.1 -sphinx-rtd-theme==1.2.2 +sphinx==8.2.3 +sphinx-rtd-theme diff --git a/src/docs/sphinx/build_guide/build_smith.rst b/src/docs/sphinx/build_guide/build_smith.rst new file mode 100644 index 0000000000..170f7d0428 --- /dev/null +++ b/src/docs/sphinx/build_guide/build_smith.rst @@ -0,0 +1,253 @@ +.. ## Copyright (c) Lawrence Livermore National Security, LLC and +.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. +.. ## +.. ## SPDX-License-Identifier: (BSD-3-Clause) + +.. _build_smith-label: + +========================= +Building Smith with CMake +========================= + +Smith uses CMake as its build system. Due to our amount of +Third-party Libraries (TPLs) and configuration options, we recommend +utilizing a :ref:`host-config ` to encapsulate most +of the build information necessary to build Smith. + +If you built the dependencies using Spack/uberenv, the host-config file is output at the +project root. If you are on LC and a member of the ``smithdev`` group, you can use +our provided host-configs in the ``host-config`` directory that follow the pattern +``--.cmake``. Contact `Brandon Talamini `_ for access. + +We provide a python script that encapsulates the CMake configuration step +but you can also use the CMake executable directly. Below are instructions +for both. + + +Option 1: Configuring the build with ``config-build.py`` +-------------------------------------------------------- + +``config-build.py`` is a python script that is aimed at simplifying and hardening +running CMake. It creates a build and install directory then runs the necessary +CMake command for you. You just need to point the script +at the generated or a provided host-config. + +``config-build.py`` has some command line options it understand to simplify the +build, they are listed in the table below. Any extra options will be passed directly +to CMake. Here are some examples on how to run ``config-build.py``: + +.. code-block:: bash + + # Just a host-config + $ ./config-build.py -hc /path/to/host-config.cmake + + # host-config + debug build + $ ./config-build.py -hc /path/to/host-config.cmake -bt Debug + + # host-config + CMake options + $ ./config-build.py -hc /path/to/host-config.cmake -DENABLE_WARNINGS_AS_ERRORS=OFF + + +.. important:: + + ``config-build.py`` automatically deletes the build and install directories. + Do **not** store any files in these directories that you wish to keep. These + directories should be treated as **temporary**, as their contents may + be removed at any time during the build process. + + +.. list-table:: ``config-build.py`` command-line options + :header-rows: 1 + :widths: 18 28 14 60 + + * - Short Option + - Long Option + - Default + - Description + + * - ``-bp`` + - ``--buildpath`` + - ``""`` + - Specify path for the build directory. If not specified, it will be created + in the current directory. + + * - ``-ip`` + - ``--installpath`` + - ``""`` + - Specify path for the install directory. If not specified, it will be created + in the current directory. + + * - ``-bt`` + - ``--buildtype`` + - ``Release`` + - Specify the CMake build type. Valid options are ``Release``, ``Debug``, + ``RelWithDebInfo``, and ``MinSizeRel``. + + * - ``-e`` + - ``--eclipse`` + - ``False`` + - Create an Eclipse project file. + + * - ``-ecc`` + - ``--exportcompilercommands`` + - ``False`` + - Generate a compilation database (``compile_commands.json``) in the build + directory. Can be used by Clang tools such as ``clang-modernize``. + + * - ``-hc`` + - ``--hostconfig`` + - ``""`` + - Select a specific host-config file to initialize CMake’s cache. + + * - *(none)* + - ``--print-default-host-config`` + - ``False`` + - Print the default host configuration for this system and exit. + + * - *(none)* + - ``--print-machine-name`` + - ``False`` + - Print the machine name for this system and exit. + + * - ``-n`` + - ``--ninja`` + - ``False`` + - Use the Ninja generator instead of Make to build the project. + + +Option 2: Configuring the build with CMake +------------------------------------------ + +Another option is to use CMake directly, this can also be useful if you configure VSCode +to build Smith. + +Here are some examples on how to run CMake: + +.. code-block:: bash + + # Just a host-config with build and install directories + $ cmake -C /path/to/host-config.cmake -B build -DCMAKE_INSTALL_PREFIX=/path/to/install + + # host-config + debug build + $ cmake -C /path/to/host-config.cmake -DCMAKE_BUILD_TYPE=Debug + + # host-config + CMake options + $ cmake -C /path/to/host-config.cmake -DENABLE_WARNINGS_AS_ERRORS=OFF + + +.. list-table:: CMake configuration options + :header-rows: 1 + :widths: 35 15 60 + + * - Option + - Default + - Description + + * - ``CMAKE_BUILD_TYPE`` + - ``Release`` + - Specifies the build type, see the `CMake docs `_ + + * - ``ENABLE_WARNINGS_AS_ERRORS`` + - ``ON`` + - Turns compiler warnings into errors. + + * - ``ENABLE_ASAN`` + - ``OFF`` + - Enable AddressSanitizer for memory checking. Supported only with + Clang or GCC; configuration will fail for other compilers. + + * - ``SMITH_ENABLE_CODEVELOP`` + - ``OFF`` + - Enable Smith’s *codevelop* build, including MFEM and Axom as CMake + subdirectories. + + * - ``SMITH_ENABLE_CODE_CHECKS`` + - ``ON`` + - Enable Smith’s code checks. + + * - ``SMITH_ENABLE_TESTS`` + - ``ON`` + - Enable Smith test builds. This option is only effective when + ``ENABLE_TESTS`` is ``ON``. + + * - ``SMITH_ENABLE_CUDA`` + - ``ON`` + - Enable Smith with CUDA support. This option is only effective when + ``ENABLE_CUDA`` is ``ON``. + + * - ``SMITH_ENABLE_HIP`` + - ``ON`` + - Enable Smith with HIP support. This option is only effective when + ``ENABLE_HIP`` is ``ON``. + + * - ``SMITH_ENABLE_OPENMP`` + - ``ON`` + - Enable Smith with OpenMP support. This option is only effective when + ``ENABLE_OPENMP`` is ``ON``. + + * - ``SMITH_ENABLE_GRETL`` + - ``ON`` + - Enable Smith with Gretl support. + + * - ``SMITH_ENABLE_CONTINUATION`` + - ``ON`` + - Enable the Continuation Solver. This option is automatically forced + to ``OFF`` when either ``SMITH_ENABLE_CUDA`` or ``SMITH_ENABLE_HIP`` + is enabled, as GPU builds are currently unsupported. + + * - ``SMITH_ENABLE_PROFILING`` + - ``OFF`` + - Enable profiling functionality. This option is automatically enabled + when benchmarking is enabled unless explicitly set by the user. + + * - ``ENABLE_BENCHMARKS`` + - ``OFF`` + - Enables Google Benchmark performance tests. + + * - ``SMITH_ENABLE_BENCHMARKS`` + - ``ON`` + - Enable Smith benchmark executables. This option is only effective + when ``ENABLE_BENCHMARKS`` is ``ON``. Benchmarking requires + ``SMITH_ENABLE_PROFILING`` to be enabled; otherwise configuration + will fail. + + * - ``SMITH_USE_VDIM_ORDERING`` + - ``ON`` + - Use ``mfem::Ordering::byVDIM`` for degree-of-freedom vectors, which + is typically faster for algebraic multigrid. When disabled, + ``byNODES`` ordering is used instead. + + +Build +----- + +Once the build has been configured, Smith can be built with one of the following commands: + +.. code-block:: bash + + # Makefile + $ cd + $ make -j16 + + # Ninja + $ cd + $ ninja -j16 + + # CMake + $ cmake --build build -- -j16 + + +We provide the following useful build targets that can be run from the build directory: + +* ``test``: Runs our unit tests +* ``docs``: Builds our documentation to the following locations: + + * Sphinx: ``build-*/src/docs/html/index.html`` + * Doxygen: ``/build-*/src/docs/html/doxygen/html/index.html`` + +* ``style``: Runs styling over source code and replaces files in place +* ``check``: Runs a set of code checks over source code (CppCheck and clang-format) +* ``install``: Installs Smith into the previously given ``CMAKE_INSTALL_PREFIX`` directory + + + diff --git a/src/docs/sphinx/build_guide/build_tpls.rst b/src/docs/sphinx/build_guide/build_tpls.rst new file mode 100644 index 0000000000..5b98c7ed4d --- /dev/null +++ b/src/docs/sphinx/build_guide/build_tpls.rst @@ -0,0 +1,177 @@ +.. ## Copyright (c) Lawrence Livermore National Security, LLC and +.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. +.. ## +.. ## SPDX-License-Identifier: (BSD-3-Clause) + +.. _build_tpls-label: + +====================================== +Building Smith's Third-party Libraries +====================================== + +It is now time to build Smith's Third-party Libraries (TPLs) and optionally our Developer Tools. + +.. _devtools-label: + +Building Developer Tools +------------------------ + +.. note:: + This section can be skipped if you are not developing Smith and contributing back to our repository. It + can also be skipped if you are a part of the ``smithdev`` LC Linux group, as it will be automatically + added by our Spack Environment. + +Smith developers utilize some industry standard development tools in their everyday work. These tools +can take a very long time to build and it is recommended to build them separately from the TPLs then +include them in the followup TPL build. Unlike Smith's library dependencies, our developer tools can be +built with any compiler because they are not linked into the smith executable. We recommend Clang 19 +because we have tested that they all build with that compiler. If you wish to build them yourself +(which takes a long time), use one of the following commands: + +For LC machines: + +.. code-block:: bash + + # These commands are equivalent, pick one + $ scripts/llnl/build_devtools.py --directory= + $ scripts/uberenv/uberenv.py --project-json=scripts/spack/devtools.json --spack-env-file=scripts/spack/devtools_configs/$SYS_TYPE/spack.yaml --prefix=../path/to/install + + +For other machines utilize the Spack Environment created in the previous :ref:`setup_system-label` step: + +.. code-block:: bash + + $ scripts/uberenv/uberenv.py --project-json=scripts/spack/devtools.json --spack-env-file= --prefix=../path/to/install + + +For example on **Ubuntu 24.04**: + +.. code-block:: bash + + $ scripts/uberenv/uberenv.py --project-json=scripts/spack/devtools.json --spack-env-file=scripts/spack/configs/linux_ubuntu_24/spack.yaml --prefix=../path/to/install + + +After the Developer Tools have been successfully installed, you have two options to use them in the following step to build +your TPLs. + +1. Alter your Spack Environment by adding the following lines with the correct paths and versions: + + .. code-block:: yaml + + cppcheck: + version: [2.9] + buildable: false + externals: + - spec: cppcheck@2.9 + prefix: /path/to/devtools_install/cppcheck-2.9 + + +2. Add a Spack upstream to the ``uberenv`` commands below with this command line option ``--upstream=../path/to/devtools_install``. + + +Building TPLs +------------- + +Run the command with the compiler that you want to develop with: + +.. code-block:: bash + + scripts/uberenv/uberenv.py --prefix= --spack-env-file= --spec="%clang_19" + +Some helpful uberenv options include: + +* ``--spec="+profiling build_type=Debug %clang_19"`` (Spack spec, ``smith@develop`` automatically prepended) +* ``--spack-env-file=`` (use specific Spack environment configuration file) +* ``--prefix=`` (required, build and install the dependencies in a particular location) - this *must be outside* of your local Smith repository + +The rest of Uberenv's command line options can be seen `here `_. + +**Basic Spack variants:** + ++-----------------+---------+---------------------------------------------------------------+ +| Variant | Default | Description | ++=================+=========+===============================================================+ +| build_type | Release | CMake build type (Debug, Release, RelWithDebInfo, MinSizeRel) | ++-----------------+---------+---------------------------------------------------------------+ +| shared | False | Enable build of shared libraries | ++-----------------+---------+---------------------------------------------------------------+ +| asan | False | Enable Address Sanitizer flags | ++-----------------+---------+---------------------------------------------------------------+ +| openmp | True | Enable OpenMP support | ++-----------------+---------+---------------------------------------------------------------+ +| devtools | False | Build development tools (such as Sphinx, CppCheck, | +| | | ClangFormat, etc...) | ++-----------------+---------+---------------------------------------------------------------+ +| cuda | False | Enable CUDA support (requires ``cuda_arch=``) | ++-----------------+---------+---------------------------------------------------------------+ +| cuda_arch | N/A | Set CUDA architecture (for example, ``70``), can be single or | +| | | comma delimited | ++-----------------+---------+---------------------------------------------------------------+ +| rocm | False | Enable ROCM support (requires ``amdgpu_target=)`` | ++-----------------+---------+---------------------------------------------------------------+ +| amdgpu_target | N/A | Set ROCM target (for example, ``gfx942``), can be single or | +| | | comma delimited | ++-----------------+---------+---------------------------------------------------------------+ + + +.. note:: + Spack variants come in two flavors: Boolean, which is indicated in the spec by ``+`` for + true and ``~`` for false, and Multi-value variants, which must start with a space and require a value + after the ``=`` sign (for example, ``+cuda cuda_arch=80``). + + +.. note:: + If you are building on LC, using our provided Spack Environments, and do not have access to the ``smithdev`` linux group, + you cannot use our prebuilt Developer Tools referenced in the Spack Environment files. You will need to turn off the + devtool variant by adding ``~devtools`` to your Spack spec via the Spack or uberenv command line. + +.. note:: + If you are building on macOS, the invocation of ``uberenv.py`` may need to be slightly modified from the + standard instructions below in order to force the use of the Homebrew-installed MPI and compilers. + The spec command line option should be ``--spec="^openmpi@5 %clang_19"`` and to build with devtools and profiling enabled, + change the spec to ``"+devtools+profiling ^openmpi@5 %clang_19"``. + + +**TPL Spack variants:** + ++-------------+---------+---------------------------------------------------------------+ +| Variant | Default | Description | ++=============+=========+===============================================================+ +| adiak | False | Build with adiak | ++-------------+---------+---------------------------------------------------------------+ +| caliper | False | Build with caliper | ++-------------+---------+---------------------------------------------------------------+ +| enzyme | False | Enable Enzyme Automatic Differentiation Framework | ++-------------+---------+---------------------------------------------------------------+ +| petsc | True | Enable PETSc support | ++-------------+---------+---------------------------------------------------------------+ +| raja | True | Build with portable kernel execution support | ++-------------+---------+---------------------------------------------------------------+ +| slepc | True | Enable SLEPc integration | ++-------------+---------+---------------------------------------------------------------+ +| strumpack | True | Build MFEM TPL with Strumpack, a direct linear solver library | ++-------------+---------+---------------------------------------------------------------+ +| sundials | True | Build MFEM TPL with SUNDIALS nonlinear/ODE solver support | ++-------------+---------+---------------------------------------------------------------+ +| tribol | True | Build Tribol, an interface physics library | ++-------------+---------+---------------------------------------------------------------+ +| umpire | True | Build with portable memory access support | ++-------------+---------+---------------------------------------------------------------+ + + +If successful, you will see two things. The first is what we call a host-config. It is all the CMake +inputs you need to build Smith. This file will be a new CMake file in the current directory with your machine +name, system type, and compiler, for example ``mycomputerlinux-ubuntu24.04-skylake-clang@19.1.1.cmake``. +The second will be output from Spack that ends in this: + +.. code-block:: bash + + ==> smith: Executing phase: 'initconfig' + +-------------- +Building Smith +-------------- + +Finally, with the TPL's built and the host-config file, you can build Smith +for more detail instructions on how to build Smith, see :ref:`build_smith-label`. + diff --git a/src/docs/sphinx/build_guide/index.rst b/src/docs/sphinx/build_guide/index.rst new file mode 100644 index 0000000000..716e41ecd7 --- /dev/null +++ b/src/docs/sphinx/build_guide/index.rst @@ -0,0 +1,129 @@ +.. ## Copyright (c) Lawrence Livermore National Security, LLC and +.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. +.. ## +.. ## SPDX-License-Identifier: (BSD-3-Clause) + +.. _build_guide-label: + +=========== +Build Guide +=========== + +This guide provides instructions on how to build or install all dependencies of Smith, followed by +how to build Smith from source. The process consists of the following high-level phases: + +#. Setup the machine by installing the required system-level packages and generating a Spack environment file +#. Build Third-party Libraries (TPLs) using Spack and Uberenv to provide a consistent dependency stack +#. Build Smith itself using CMake once all dependencies are in place + +.. note:: + On LC machines, it is good practice to run intensive build commands in parallel on a compute node. + Here is an example SLURM command, ``srun -ppdebug -N1 --exclusive make -j16`` and an example + Flux command, ``flux run -qpdebug -N1 --exclusive make -j16``. + +.. tip:: + + If you are on a LC machine and are in the ``smithdev`` LC linux group, we have a variety of pre-installed + TPLs and configurations. If these are sufficient for you, you can skip to :ref:`build_smith-label`. + You can see these machines and configurations in the ``host-configs`` repository directory. + +.. tip:: + + Alternatively, you can build Smith using preinstalled dependencies inside our existing Docker + containers. This may have runtime speed considerations. Instructions for this process are + located :ref:`here `. + +.. note:: + + Smith uses the LLVM plugin `Enzyme `_ to perform + automatic differentiation. To enable this functionality, you have to compile with an + LLVM-based compiler. We recommend ``clang``. + +.. toctree:: + :hidden: + :titlesonly: + :maxdepth: 2 + + setup_system/index + build_tpls + build_smith + + +Spack and Uberenv +----------------- + +Smith uses `Spack `_ to build its TPLs. +This has been encapsulated using `Uberenv `_. +Uberenv helps by doing the following: + +* Pulls a blessed version of Spack and Spack Packages locally +* If you are on a known operating system (like TOSS4), Uberenv will automatically + use our blessed Spack Environment files to keep Spack from building the world +* Installs our Spack packages into the local Spack installation location +* Simplifies whole dependency build into one command + +Uberenv will create a directory containing a Spack instance with the required Smith +TPLs installed. + +.. note:: + This directory **must not** be within the Smith repo - the example below + controls this with the ``--prefix`` command line argument which is required. + + +.. _host_config-label: + +Host-configs +------------ + +Our Spack package recipe generates a file we call a host-config (``.cmake``) +at the root of Smith repository. CMake refers to this file as an +`initial cache `_. +This host-config defines all the required information for building Smith including paths to compilers, +TPLs, Developer Tools, and machine specific options. + + +Cloning Smith +------------- + +Smith is hosted on `GitHub `_. Smith uses git submodules, so the project must be cloned +recursively. Use either of the following commands to pull Smith's repository: + +.. code-block:: bash + + # Using SSH keys setup with GitHub + $ git clone --recursive git@github.com:LLNL/smith.git + + # Using HTTPS which works for everyone but is slightly slower and will require username/password + # for some commands + $ git clone --recursive https://github.com/LLNL/smith.git + + +Phase 1: Basic System Setup +--------------------------- + +We recommend installing some basic system-level development packages to minimize the +amount of packages that Spack will build. + +The following pages provide basic guidance on the following platforms and is where you should +start: + +* :ref:`Livermore Computing (LC) ` +* :ref:`macOS ` +* :ref:`Ubuntu 24 ` + +At the end of each Setup guide, it has a link to the page that shows you how to build +the minimal set of TPLs for Smith; followed by a page on how to build +Smith from the generated host-config file via CMake. + + +Phase 2: Build Third-party Libraries +------------------------------------ + +For more information see :ref:`build_tpls-label`. + + +Phase 3: Build Smith with CMake +------------------------------- + +For more information see :ref:`build_smith-label`. + diff --git a/src/docs/sphinx/build_guide/setup_system/index.rst b/src/docs/sphinx/build_guide/setup_system/index.rst new file mode 100644 index 0000000000..3cc01d9f23 --- /dev/null +++ b/src/docs/sphinx/build_guide/setup_system/index.rst @@ -0,0 +1,33 @@ +.. ## Copyright (c) Lawrence Livermore National Security, LLC and +.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. +.. ## +.. ## SPDX-License-Identifier: (BSD-3-Clause) + +.. _setup_system-label: + +================== +Basic System Setup +================== + +We recommend installing some basic system-level development packages to minimize the +amount of packages that Spack will build. + +The following pages provide basic guidance on the following platforms and is where you should +start: + +* :ref:`Livermore Computing (LC) ` +* :ref:`macOS ` +* :ref:`Ubuntu 24 ` + +At the end of each Setup guide, it has a link to the page that shows you how to build +the minimal set of TPLs for Smith; followed by a page on how to build +Smith from the generated host-config file via CMake. + +.. toctree:: + :hidden: + :maxdepth: 1 + + setup_lc + setup_macos + setup_ubuntu + diff --git a/src/docs/sphinx/build_guide/setup_system/setup_lc.rst b/src/docs/sphinx/build_guide/setup_system/setup_lc.rst new file mode 100644 index 0000000000..479cb27036 --- /dev/null +++ b/src/docs/sphinx/build_guide/setup_system/setup_lc.rst @@ -0,0 +1,119 @@ +.. ## Copyright (c) Lawrence Livermore National Security, LLC and +.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. +.. ## +.. ## SPDX-License-Identifier: (BSD-3-Clause) + +.. _setup_lc-label: + +======================================== +Setup Livermore Computing (LC) TPL Build +======================================== + +We provide Spack Environment files for each of LC's systems: + + * TOSS4: ``scripts/spack/configs/toss_4_x86_64_ib/spack.yaml`` + * TOSS4 Cray: ``scripts/spack/configs/toss_4_x86_64_ib_cray/spack.yaml`` + +Unless otherwise specified, Spack will default to a compiler. This is generally not a good idea when +developing large codes. To specify which compiler to use add the compiler specification to the ``--spec`` Uberenv +command line option. We provide recommended Spack specs for LC in ``scripts/spack/specs.json``. + +You can use these directly in the ``uberenv.py`` command in the :ref:`build_tpls-label` +section by substituting the values in these two command line options: ``--spack-env-file=ubuntu24.yaml --spec="%clang_19"``. + +.. note:: + On LC machines, it is good practice to do the build step in parallel on a compute node. + You should add the following to the start of your commands: ``salloc -ppdebug -N1 --exclusive python3 scripts/uberenv/uberenv.py`` + +.. note:: + If you do not have access to the ``smithdev`` linux group, you cannot use our prebuilt Developer Tools + referenced in the Spack Environment files listed above. You will need to turn off the devtool variant + by adding ``~devtools`` to your Spack spec via the Spack or uberenv command line. + +------------------------------- +Generate Spack Environment File +------------------------------- + +Spack uses an environment file, or ``spack.yaml``, to describe where system level packages are to minimize what it builds. +This file describes the compilers and associated flags required for the platform as well as the low-level libraries +on the system to prevent Spack from building the world. Documentation on these environment files is located +in the `Spack docs `_. + +The following command will download the specific Spack version we use and run a minimal set of commands to +generate an environment file for you. This should be a good starting point and should be used in following +Spack builds. + +.. code-block:: bash + + ./scripts/uberenv/uberenv.py --prefix= --setup-and-env-only + + +The above spack command will output a concretization that looks like the following: + +.. code-block:: shell + + ==> Concretized 1 spec: + - uu3sgzv smith@develop cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~asan~cuda~devtools+enzyme~ipo+openmp+petsc~profiling+raja~rocm~shared+slepc+strumpack+sundials+tribol+umpire build_system=cmake build_type=Release dev_path=/home/white238/projects/smith/repo generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - x77izrn ^axom@0.10.1.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +cpp14~cuda~devtools~examples~fortran+hdf5~ipo+lua+mfem+mpi~opencascade+openmp~python+raja~rocm~scr~shared~tools+umpire build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - prysqkw ^blt@0.6.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=generic arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 6vi46wm ^camp@2024.07.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cuda~ipo~omptarget+openmp~rocm~sycl~tests build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] fbcccfh ^cmake@3.28.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release patches=dbc3892 arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - zqg3svf ^conduit@0.9.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~adios+blt_find_mpi~caliper~doc~doxygen+examples~fortran+hdf5+hdf5_compat~ipo+mpi+parmetis~python+shared~silo~test+utilities~zfp build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - lbmoj2n ^enzyme@0.0.180 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~ipo build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] 6bbmbqw ^llvm@19.1.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +clang~cuda~flang+gold+libomptarget~libomptarget_debug~link_llvm_dylib~lld~lldb+llvm_dylib+lua~mlir+offload+polly~python~split_dwarf~z3~zstd build_system=cmake build_type=Release compiler-rt=runtime generator=ninja libcxx=runtime libunwind=runtime openmp=runtime shlib_symbol_version=none targets=all version_suffix=none arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] wyizjq2 ^glibc@2.39 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] 74zxzg7 ^gmake@4.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~guile build_system=generic patches=599f134 arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 32xbf3o ^hdf5@1.8.23 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cxx~fortran+hl~ipo~mpi~shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=f42732a arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - hlqkvfc ^pkgconf@2.3.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - dlqs5c6 ^zlib-ng@2.2.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +compat~new_strategies+opt+pic+shared build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - iwu2tah ^hypre@2.32.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~caliper~complex~cublas~cuda~debug+fortran~gptune~gpu-aware-mpi~int64~internal-superlu+lapack~magma~mixedint+mpi~openmp~rocblas~rocm~shared~superlu-dist~sycl~umpire~unified-memory build_system=autotools precision=double arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - g6pkuqj ^openblas@0.3.29 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared build_system=makefile patches=9968625 symbol_suffix=none threads=openmp arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] naati2q ^lua@5.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +shared build_system=makefile fetcher=curl arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 5u3dj5i ^metis@5.1.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~gdb~int64~ipo~no_warning~real64~shared build_system=cmake build_type=Release generator=make patches=4991da9,93a7903 arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - ae2mqqr ^mfem@4.8.0.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~amgx~asan~conduit~cuda~debug~examples~exceptions~fms~ginkgo~gnutls~gslib~hiop+lapack~libceed~libunwind+metis~miniapps~mpfr+mpi~mumps+netcdf~occa+openmp+petsc~pumi~raja~rocm~shared+slepc+static+strumpack~suite-sparse+sundials+superlu-dist~threadsafe~umpire+zlib build_system=generic cxxstd=auto precision=double timer=auto arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - ohbx2dl ^netcdf-c@4.7.4 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~blosc~byterange~dap~fsync~hdf4~jna~logging~mpi~nczarr_zip+optimize~parallel-netcdf+pic~shared~szip~zstd build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] yr5mrlv ^openmpi@4.1.6 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +atomics~cuda+cxx~cxx_exceptions~debug~gpfs~internal-hwloc~internal-libevent~internal-pmix+java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix+pmi+romio+rsh~singularity~static~two_level_namespace+vt~wrapper-rpath build_system=autotools fabrics=ofi,psm,psm2,ucx romio-filesystem=none schedulers=slurm arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 2zrmzi6 ^parmetis@4.0.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~gdb~int64~ipo~shared build_system=cmake build_type=Release generator=make patches=4f89253,50ed208,704b84f arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - sxzqrlk ^petsc@3.22.4 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~X~batch~cgns~complex~cuda~debug+double~exodusii~fftw+fortran~giflib~hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack~shared+strumpack~suite-sparse+superlu-dist~sycl~tetgen~trilinos~valgrind~zoltan build_system=generic clanguage=C memalign=none arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] baweecy ^diffutils@3.10 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - evwepln ^netlib-scalapack@2.2.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~ipo~pic+shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] timms67 ^python@3.12.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat~pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - cc22mbv ^raja@2024.07.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cuda~desul~examples~exercises~ipo~lowopttest~omptarget~omptask+openmp~plugins~rocm~run-all-tests~shared~sycl~tests+vectorization build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - zmiasfe ^slepc@3.22.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +arpack~blopex~cuda~hpddm~rocm build_system=generic arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - fvqm3cm ^arpack-ng@3.9.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~icb~ipo+mpi+shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 3ka3cml ^strumpack@8.0.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~butterflypack+c_interface~count_flops~cuda~ipo~magma+mpi+openmp+parmetis~rocm~scotch~shared~slate~task_timers~zfp build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - mkzronq ^sundials@6.7.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +ARKODE+CVODE+CVODES+IDA+IDAS+KINSOL~asan~cuda~examples~examples-install~f2003~fcmix+generic-math~ginkgo+hypre~int64~ipo~klu~kokkos~kokkos-kernels~lapack~magma~monitoring+mpi~openmp~petsc~profiling~pthread~raja~rocm~shared+static~superlu-dist~superlu-mt~sycl~trilinos build_system=cmake build_type=Release cstd=99 cxxstd=14 generator=make logging-level=2 logging-mpi=OFF precision=double arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - vqgxdhb ^superlu-dist@8.1.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cuda~int64~ipo~openmp+parmetis~rocm~shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - ldxkes6 ^tribol@0.1.0.18 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~asan~cuda~devtools~examples~fortran~ipo+openmp+raja+redecomp~rocm~tests+umpire build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 3ftpxxz ^umpire@2024.07.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~asan~backtrace+c~cuda~dev_benchmarks~device_alloc~deviceconst~examples+fmt_header_only~fortran~ipc_shmem~ipo~mpi~numa~omptarget+openmp~rocm~sanitizer_tests~shared~sqlite_experimental~tools+werror build_system=cmake build_type=Release generator=make tests=none arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - xts3eqq ^fmt@11.0.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~ipo+pic~shared build_system=cmake build_type=Release cxxstd=11 generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + + +To speed up the build, you can add packages that exist on your system and were not detected by Spack to the same +Spack environment file. For example, there is a lua installed in ``/usr/lib64`` but if it was not found by Spack, +you could add the following lines under the ``packages:`` section of the yaml: + +.. code-block:: yaml + + lua: + externals: + - spec: lua@5.2 + prefix: /usr + buildable: false + + +Lines starting with ``[e]`` are external packages that Spack recognizes are on the system and will not rebuild them. +By adding Lua to the Spack environment file, Spack will no longer build Lua and any of its dependencies that are +needed by anything else. In this example adding an external ``lua``, removes the need to build ``readline`` and ``unzip``. + +.. important:: + + Uberenv will override existing ``spack.yaml`` files in the current working directory. Now that we have made modifications, + you should rename/move the file so the changes are not lost and adjust the `uberenv.py` commands to reflect the new file name. + +-------------------------------------- +Building Smith's Third-party Libraries +-------------------------------------- + +It is now time to build Smith's Third-party Libraries (TPLs). For detailed instructions see :ref:`build_tpls-label`. + diff --git a/src/docs/sphinx/dev_guide/tpl_build.rst b/src/docs/sphinx/build_guide/setup_system/setup_macos.rst similarity index 69% rename from src/docs/sphinx/dev_guide/tpl_build.rst rename to src/docs/sphinx/build_guide/setup_system/setup_macos.rst index 935804906f..219fb7ea0c 100644 --- a/src/docs/sphinx/dev_guide/tpl_build.rst +++ b/src/docs/sphinx/build_guide/setup_system/setup_macos.rst @@ -3,102 +3,16 @@ .. ## .. ## SPDX-License-Identifier: (BSD-3-Clause) -.. _tpl_builds-label: +.. _setup_macos-label: -=============================== -Third-party Library (TPL) Build -=============================== - ------------------ -Spack and Uberenv ------------------ - -Smith uses `Spack `_ to build it's TPLs. -This has been encapsulated using `Uberenv `_. -Uberenv helps by doing the following: - -* Pulls a blessed version of Spack locally -* If you are on a known operating system (like TOSS4), we have defined Spack configuration files - to keep Spack from building the world -* Installs our Spack packages into the local Spack Environment -* Simplifies whole dependency build into one command - -Uberenv will create a directory containing a Spack instance with the required Smith -TPLs installed. - -.. note:: - This directory **must not** be within the Smith repo - the example below - controls this with the ``--prefix`` command line argument which is required. - -It also generates a host-config file (``.cmake``) -at the root of Smith repository. This host-config defines all the required information for building -Smith. - ------------------- -Basic System Setup ------------------- - -We recommend installing some basic system-level development packages to minimize the -amount of packages that Spack will build. - -The following page provides basic guidance on the following platforms: - - * Ubuntu 24 LTS - * macOS - * Livermore Computing (LC) - -.. note:: - - Smith uses the LLVM plugin `Enzyme `_ to perform - automatic differentiation. Due to this you have to compile with an LLVM-based compiler. - We recommend ``clang``. - -Ubuntu 24 LTS -^^^^^^^^^^^^^ - -Install clang version 19 and make it the default compiler: - -.. code-block:: bash - - sudo apt install -y --no-install-recommends clang-19 libclang-19-dev clang-format-19 llvm-19 llvm-19-dev libzstd-dev libomp-19-dev gfortran-13 - # Set clang-19 as the default clang - sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-19 101 \ - && sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-19 101 \ - && sudo update-alternatives --install /usr/bin/clang-format clang-format /usr/bin/clang-format-19 101 - -Install required build packages to minimize what Spack will build: - -.. code-block:: bash - - sudo apt install -y --no-install-recommends build-essential bzip2 cmake libopenblas-dev \ - lua5.2 liblua5.2-dev openmpi-bin libopenmpi-dev unzip - -Optionally you can install packages to generate documentation: - -.. code-block:: bash - - sudo apt install -y --no-install-recommends graphviz python3-sphinx texlive-full doxygen - -.. note:: - - The documentation packages require a lot of disk space. - -.. note:: - - We provide a basic Ubuntu 24 Spack environment file in ``scripts/spack/configs/linux_ubuntu_24`` that - may work for most people. If you want to try using that, skip to :ref:`building_tpls-label` - below and use this command line option instead ``--spack-env-file=scripts/spack/configs/linux_ubuntu_24/spack.yaml`` - -macOS -^^^^^ - -.. warning:: - These instructions are in development, but have been tested for M2 MacBooks. +===================== +Setup macOS TPL Build +===================== .. note:: View an example host-config for MacOS in ``host-configs/other/firion-macos_sonoma_aarch64-.cmake``. -Homebrew is recommended to install base dependencies due to it's stability. Relying on pure Spack historically leads to more failed builds. +Homebrew is recommended to install base dependencies. Relying on pure Spack historically leads to more failed builds. To start, install the following packages using Homebrew. @@ -130,16 +44,11 @@ This is also useful for a few additional packages: .. note:: We provide a basic MacOS Spack environment file that - may work for most people. If you want to try using that, skip to :ref:`building_tpls-label` + may work for most people. If you want to try using that, skip to :ref:`build_tpls-label` below and use this command line option instead ``--spack-env-file=scripts/spack/configs/darwin/spack.yaml``. You will likely need to update the versions of packages to match the versions installed by Homebrew. The versions for all installed packages can be listed via the command ``brew list --versions``. -.. note:: - The invocation of ``uberenv.py`` is slightly modified from the standard instructions below - in order to force the use of the Homebrew-installed MPI and compilers. The spec command line option - should be ``--spec="^openmpi@5 %clang_19"`` and to build with devtools and profiling enabled, - change the spec to ``"+devtools+profiling ^openmpi@5 %clang_19"`` Given that Homebrew can only install CMake version 4.0 and it breaks some TPL builds (e.g. metis), its recommended to install an older version of CMake manually. You can do this by downloading from `CMake's official archive `_. After installing @@ -192,30 +101,6 @@ Versions and prefixes may vary. - spec: py-sphinx@7.4.7 prefix: /path/to/venv -Livermore Computing (LC) -^^^^^^^^^^^^^^^^^^^^^^^^ - -We provide Spack Environment files for each of LC's systems: - - * TOSS4: ``scripts/spack/configs/toss_4_x86_64_ib/spack.yaml`` - * BlueOS: ``scripts/spack/configs/blueos_3_ppc64le_p9/spack.yaml`` - * TOSS4 Cray: ``scripts/spack/configs/toss_4_x86_64_ib_cray/spack.yaml`` - -Unless otherwise specified, Spack will default to a compiler. This is generally not a good idea when -developing large codes. To specify which compiler to use add the compiler specification to the ``--spec`` Uberenv -command line option. We provide recommended Spack specs for LC in ``scripts/spack/specs.json``. - -You can use these directly in the ``uberenv.py`` command in the :ref:`building_tpls-label` -section by substituting the values in these two command line options: ``--spack-env-file=ubuntu24.yaml --spec="%clang_19"``. - -.. note:: - On LC machines, it is good practice to do the build step in parallel on a compute node. - You should add the following to the start of your commands: ``salloc -ppdebug -N1 --exclusive python3 scripts/uberenv/uberenv.py`` - -.. note:: - If you do not have access to the ``smithdev`` linux group. You cannot currently use our prebuilt Dev Tools - referenced in the Spack Environment files listed above. You will be required to turn off the devtool variant - on your Spack spec by adding ``~devtools`` to your uberenv or Spack spec. ------------------------------- Generate Spack Environment File @@ -256,14 +141,15 @@ If you want to use Clang as your compiler, alter the following section in that f To speed up the build, you can add packages that exist on your system to the same Spack environment file. For example, -we installed lua in the above ``apt`` commands. To do so, add the following lines under the ``packages:`` section of the yaml: +you can install lua via homebrew with this command, ``brew install lua``. Then you can add it as a Spack external +in the ``packages:`` section of the Spack Environment yaml file: .. code-block:: yaml lua: externals: - spec: lua@5.2 - prefix: /usr + prefix: /opt/homebrew buildable: false The above spack command will output a concretization that looks like the following: @@ -325,57 +211,9 @@ by another package, so you can also add it with this yaml section: Uberenv will override existing ``spack.yaml`` files in the current working directory. Now that we have made modifications, you should rename/move the file so the changes are not lost and adjust the `uberenv.py` commands to reflect the new file name. - -.. _building_tpls-label: - -------------------------------------- Building Smith's Third-party Libraries -------------------------------------- -It is now time to build Smith's Third-party Libraries (TPLs). Run the command with the compiler -that you want to develop with: - -.. code-block:: bash - - scripts/uberenv/uberenv.py --prefix= --spack-env-file= --spec="%clang_19" - -Some helpful uberenv options include : - -* ``--spec=" build_type=Debug"`` (build core TPLs, such as MFEM and Hypre, with debug symbols) -* ``--spec=+profiling`` (build the Adiak and Caliper libraries) -* ``--spec=+devtools`` (also build the devtools with one command) -* ``--spec=%clang_19`` (build with a specific compiler as defined in the ``spack.yaml`` file) -* ``--spack-env-file=`` (use specific Spack environment configuration file) -* ``--prefix=`` (required, build and install the dependencies in a particular location) - this *must be outside* of your local Smith repository - -The modifiers to the Spack specification ``spec`` can be chained together, e.g. ``--spec='+devtools build_type=Debug %clang_19'``. - - -If successful, you will see two things. The first is what we call a host-config. It is all the CMake -inputs you need to build Smith. This file will be a new CMake file in the current directory with your machine -name, system type, and compiler, for example ``mycomputerlinux-ubuntu24.04-skylake-clang@19.1.1.cmake``. -The second will be output from Spack that ends in this: - -.. code-block:: bash - - ==> smith: Executing phase: 'initconfig' - ==> Updating view at /my/prefix/spack_env/.spack-env/view - --------------- -Building Smith --------------- - -Finally, with the TPL's built and the host-config file, you can build Smith with the following -command: - -.. code-block:: bash - - ./config-build.py -hc - cd - make -j - make -j8 test - -For more detail instructions on how to build Smith, see :ref:`build-label`. - - +It is now time to build Smith's Third-party Libraries (TPLs). For detailed instructions see :ref:`build_tpls-label`. diff --git a/src/docs/sphinx/build_guide/setup_system/setup_ubuntu.rst b/src/docs/sphinx/build_guide/setup_system/setup_ubuntu.rst new file mode 100644 index 0000000000..c4b75d8e83 --- /dev/null +++ b/src/docs/sphinx/build_guide/setup_system/setup_ubuntu.rst @@ -0,0 +1,160 @@ +.. ## Copyright (c) Lawrence Livermore National Security, LLC and +.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. +.. ## +.. ## SPDX-License-Identifier: (BSD-3-Clause) + +.. _setup_ubuntu-label: + +========================= +Setup Ubuntu 24 TPL Build +========================= + + +Install clang version 19 and make it the default compiler: + +.. code-block:: bash + + sudo apt install -y --no-install-recommends clang-19 libclang-19-dev clang-format-19 llvm-19 llvm-19-dev libzstd-dev libomp-19-dev gfortran-13 + # Set clang-19 as the default clang + sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-19 101 \ + && sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-19 101 \ + && sudo update-alternatives --install /usr/bin/clang-format clang-format /usr/bin/clang-format-19 101 + +Install required build packages to minimize what Spack will build: + +.. code-block:: bash + + sudo apt install -y --no-install-recommends build-essential bzip2 cmake libopenblas-dev \ + lua5.2 liblua5.2-dev openmpi-bin libopenmpi-dev unzip + +Optionally you can install packages to generate documentation: + +.. code-block:: bash + + sudo apt install -y --no-install-recommends graphviz python3-sphinx texlive-full doxygen + +.. note:: + + The documentation packages require a lot of disk space. + +.. note:: + + We provide a basic Ubuntu 24 Spack environment file in ``scripts/spack/configs/linux_ubuntu_24`` that + may work for most people. If you want to try using that, skip to :ref:`build_tpls-label` + below and use this command line option instead ``--spack-env-file=scripts/spack/configs/linux_ubuntu_24/spack.yaml`` + + +------------------------------- +Generate Spack Environment File +------------------------------- + +Spack uses an environment file, or ``spack.yaml``, to describe where system level packages are to minimize what it builds. +This file describes the compilers and associated flags required for the platform as well as the low-level libraries +on the system to prevent Spack from building the world. Documentation on these environment files is located +in the `Spack docs `_. + +The following command will download the specific Spack version we use and run a minimal set of commands to +generate an environment file for you. This should be a good starting point and should be used in following +Spack builds. + +.. code-block:: bash + + ./scripts/uberenv/uberenv.py --prefix= --setup-and-env-only + +This command will create a Spack environment file, ``spack.yaml``, where you ran the above command. +If you want to use Clang as your compiler, alter the following section in that file by changing +``null`` in the ``f77`` and ``fc`` lines to ``/usr/bin/gfortran``: + +.. code-block:: yaml + + - compiler: + spec: clang@=19.1.1 + paths: + cc: /usr/bin/clang + cxx: /usr/bin/clang++ + f77: null # Change null to /usr/bin/gfortran + fc: null # and this one too + flags: {} + operating_system: ubuntu24.04 + target: x86_64 + modules: [] + environment: {} + extra_rpaths: [] + + +To speed up the build, you can add packages that exist on your system to the same Spack environment file. For example, +we installed lua in the above ``apt`` commands. To do so, add the following lines under the ``packages:`` section of the yaml: + +.. code-block:: yaml + + lua: + externals: + - spec: lua@5.2 + prefix: /usr + buildable: false + +The above spack command will output a concretization that looks like the following: + +.. code-block:: shell + + ==> Concretized 1 spec: + - uu3sgzv smith@develop cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~asan~cuda~devtools+enzyme~ipo+openmp+petsc~profiling+raja~rocm~shared+slepc+strumpack+sundials+tribol+umpire build_system=cmake build_type=Release dev_path=/home/white238/projects/smith/repo generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - x77izrn ^axom@0.10.1.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +cpp14~cuda~devtools~examples~fortran+hdf5~ipo+lua+mfem+mpi~opencascade+openmp~python+raja~rocm~scr~shared~tools+umpire build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - prysqkw ^blt@0.6.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=generic arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 6vi46wm ^camp@2024.07.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cuda~ipo~omptarget+openmp~rocm~sycl~tests build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] fbcccfh ^cmake@3.28.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release patches=dbc3892 arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - zqg3svf ^conduit@0.9.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~adios+blt_find_mpi~caliper~doc~doxygen+examples~fortran+hdf5+hdf5_compat~ipo+mpi+parmetis~python+shared~silo~test+utilities~zfp build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - lbmoj2n ^enzyme@0.0.180 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~ipo build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] 6bbmbqw ^llvm@19.1.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +clang~cuda~flang+gold+libomptarget~libomptarget_debug~link_llvm_dylib~lld~lldb+llvm_dylib+lua~mlir+offload+polly~python~split_dwarf~z3~zstd build_system=cmake build_type=Release compiler-rt=runtime generator=ninja libcxx=runtime libunwind=runtime openmp=runtime shlib_symbol_version=none targets=all version_suffix=none arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] wyizjq2 ^glibc@2.39 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] 74zxzg7 ^gmake@4.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~guile build_system=generic patches=599f134 arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 32xbf3o ^hdf5@1.8.23 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cxx~fortran+hl~ipo~mpi~shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=f42732a arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - hlqkvfc ^pkgconf@2.3.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - dlqs5c6 ^zlib-ng@2.2.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +compat~new_strategies+opt+pic+shared build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - iwu2tah ^hypre@2.32.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~caliper~complex~cublas~cuda~debug+fortran~gptune~gpu-aware-mpi~int64~internal-superlu+lapack~magma~mixedint+mpi~openmp~rocblas~rocm~shared~superlu-dist~sycl~umpire~unified-memory build_system=autotools precision=double arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - g6pkuqj ^openblas@0.3.29 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared build_system=makefile patches=9968625 symbol_suffix=none threads=openmp arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] naati2q ^lua@5.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +shared build_system=makefile fetcher=curl arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 5u3dj5i ^metis@5.1.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~gdb~int64~ipo~no_warning~real64~shared build_system=cmake build_type=Release generator=make patches=4991da9,93a7903 arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - ae2mqqr ^mfem@4.8.0.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~amgx~asan~conduit~cuda~debug~examples~exceptions~fms~ginkgo~gnutls~gslib~hiop+lapack~libceed~libunwind+metis~miniapps~mpfr+mpi~mumps+netcdf~occa+openmp+petsc~pumi~raja~rocm~shared+slepc+static+strumpack~suite-sparse+sundials+superlu-dist~threadsafe~umpire+zlib build_system=generic cxxstd=auto precision=double timer=auto arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - ohbx2dl ^netcdf-c@4.7.4 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~blosc~byterange~dap~fsync~hdf4~jna~logging~mpi~nczarr_zip+optimize~parallel-netcdf+pic~shared~szip~zstd build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] yr5mrlv ^openmpi@4.1.6 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +atomics~cuda+cxx~cxx_exceptions~debug~gpfs~internal-hwloc~internal-libevent~internal-pmix+java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix+pmi+romio+rsh~singularity~static~two_level_namespace+vt~wrapper-rpath build_system=autotools fabrics=ofi,psm,psm2,ucx romio-filesystem=none schedulers=slurm arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 2zrmzi6 ^parmetis@4.0.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~gdb~int64~ipo~shared build_system=cmake build_type=Release generator=make patches=4f89253,50ed208,704b84f arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - sxzqrlk ^petsc@3.22.4 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~X~batch~cgns~complex~cuda~debug+double~exodusii~fftw+fortran~giflib~hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack~shared+strumpack~suite-sparse+superlu-dist~sycl~tetgen~trilinos~valgrind~zoltan build_system=generic clanguage=C memalign=none arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] baweecy ^diffutils@3.10 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC build_system=autotools arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - evwepln ^netlib-scalapack@2.2.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~ipo~pic+shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + [e] timms67 ^python@3.12.3 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat~pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - cc22mbv ^raja@2024.07.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cuda~desul~examples~exercises~ipo~lowopttest~omptarget~omptask+openmp~plugins~rocm~run-all-tests~shared~sycl~tests+vectorization build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - zmiasfe ^slepc@3.22.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +arpack~blopex~cuda~hpddm~rocm build_system=generic arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - fvqm3cm ^arpack-ng@3.9.1 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~icb~ipo+mpi+shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 3ka3cml ^strumpack@8.0.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~butterflypack+c_interface~count_flops~cuda~ipo~magma+mpi+openmp+parmetis~rocm~scotch~shared~slate~task_timers~zfp build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - mkzronq ^sundials@6.7.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC +ARKODE+CVODE+CVODES+IDA+IDAS+KINSOL~asan~cuda~examples~examples-install~f2003~fcmix+generic-math~ginkgo+hypre~int64~ipo~klu~kokkos~kokkos-kernels~lapack~magma~monitoring+mpi~openmp~petsc~profiling~pthread~raja~rocm~shared+static~superlu-dist~superlu-mt~sycl~trilinos build_system=cmake build_type=Release cstd=99 cxxstd=14 generator=make logging-level=2 logging-mpi=OFF precision=double arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - vqgxdhb ^superlu-dist@8.1.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~cuda~int64~ipo~openmp+parmetis~rocm~shared build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - ldxkes6 ^tribol@0.1.0.18 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~asan~cuda~devtools~examples~fortran~ipo+openmp+raja+redecomp~rocm~tests+umpire build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - 3ftpxxz ^umpire@2024.07.0 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~asan~backtrace+c~cuda~dev_benchmarks~device_alloc~deviceconst~examples+fmt_header_only~fortran~ipc_shmem~ipo~mpi~numa~omptarget+openmp~rocm~sanitizer_tests~shared~sqlite_experimental~tools+werror build_system=cmake build_type=Release generator=make tests=none arch=linux-ubuntu24.04-skylake %clang@19.1.1 + - xts3eqq ^fmt@11.0.2 cflags=-fPIC cxxflags=-fPIC fflags=-fPIC ~ipo+pic~shared build_system=cmake build_type=Release cxxstd=11 generator=make arch=linux-ubuntu24.04-skylake %clang@19.1.1 + + +Lines starting with ``[e]`` are external packages that Spack recognizes are on the system and will not rebuild them. +By adding Lua to the Spack environment file, Spack will no longer build Lua and any of its dependencies that are +needed by anything else. In this case, ``lua``, ``readline``, and ``unzip`` will not be built. ``unzip`` may be needed +by another package, so you can also add it with this yaml section: + +.. code-block:: yaml + + unzip: + externals: + - spec: unzip@6.0 + prefix: /usr + buildable: false + +.. important:: + + Uberenv will override existing ``spack.yaml`` files in the current working directory. Now that we have made modifications, + you should rename/move the file so the changes are not lost and adjust the `uberenv.py` commands to reflect the new file name. + + +-------------------------------------- +Building Smith's Third-party Libraries +-------------------------------------- + +It is now time to build Smith's Third-party Libraries (TPLs). For detailed instructions see :ref:`build_tpls-label`. diff --git a/src/docs/sphinx/dev_guide/docker_env.rst b/src/docs/sphinx/dev_guide/docker_env.rst index b3da247d0b..5ac8be6a21 100644 --- a/src/docs/sphinx/dev_guide/docker_env.rst +++ b/src/docs/sphinx/dev_guide/docker_env.rst @@ -34,7 +34,7 @@ If you haven't used Docker before, it is recommended that you check out the The ``-v`` option to ``docker run`` mounts a `Docker volume `_ into the container. This means that part of your filesystem (in this case, your copy of the Smith repo) will be accessible from the container. -6. Follow the build instructions detailed in the :ref:`quickstart guide `, using the host-config in ``host-configs/docker`` that +6. Follow the build instructions detailed in :ref:`build_smith-label`, using the host-config in ``host-configs/docker`` that corresponds to the compiler you've selected. These commands should be run using the terminal you opened in the previous step. Due to issues with the docker bind-mount permissions, it is suggested that you set the build and install directories to be outside of the repository. diff --git a/src/docs/sphinx/dev_guide/index.rst b/src/docs/sphinx/dev_guide/index.rst index 2f044291a1..37f4abf53b 100644 --- a/src/docs/sphinx/dev_guide/index.rst +++ b/src/docs/sphinx/dev_guide/index.rst @@ -11,7 +11,6 @@ Developer Guide :hidden: :maxdepth: 2 - tpl_build.rst style_guide docker_env modern_cpp diff --git a/src/docs/sphinx/dev_guide/macmini.rst b/src/docs/sphinx/dev_guide/macmini.rst index ca9ffadec8..f4c737950b 100644 --- a/src/docs/sphinx/dev_guide/macmini.rst +++ b/src/docs/sphinx/dev_guide/macmini.rst @@ -50,8 +50,7 @@ how to do this, instructions can be found on 7. **Install Smith** -You're now able to clone Smith and get started with the installation process. Further instructions for doing so are currently on -the `quickstart page `_ of the Smith documentation. +You're now able to clone Smith and get started with the installation process. For further instructions see :ref:`build_smith-label`. ---------------- Cron Job Example diff --git a/src/docs/sphinx/dev_guide/testing.rst b/src/docs/sphinx/dev_guide/testing.rst index 99a80ee265..6b1ff4a709 100644 --- a/src/docs/sphinx/dev_guide/testing.rst +++ b/src/docs/sphinx/dev_guide/testing.rst @@ -45,7 +45,7 @@ Requirements: used as reference. #. **Build the code.** - Build code with the normal steps. More info in the :ref:`quickstart-label`. + Build code with the normal steps. More info in the :ref:`build_smith-label`. This generates a script in the build directory called ``ats.sh``. #. **Run integration tests.** diff --git a/src/docs/sphinx/quickstart.rst b/src/docs/sphinx/quickstart.rst deleted file mode 100644 index 9d4bc17c59..0000000000 --- a/src/docs/sphinx/quickstart.rst +++ /dev/null @@ -1,151 +0,0 @@ -.. ## Copyright (c) Lawrence Livermore National Security, LLC and -.. ## other Smith Project Developers. See the top-level COPYRIGHT file for details. -.. ## -.. ## SPDX-License-Identifier: (BSD-3-Clause) - -.. _quickstart-label: - -====================== -Quickstart Guide -====================== - -Getting Smith -------------- - -Smith is hosted on `GitHub `_. Smith uses git submodules, so the project must be cloned -recursively. Use either of the following commands to pull Smith's repository: - -.. code-block:: bash - - # Using SSH keys setup with GitHub - $ git clone --recursive git@github.com:LLNL/smith.git - - # Using HTTPS which works for everyone but is slightly slower and will require username/password - # for some commands - $ git clone --recursive https://github.com/LLNL/smith.git - -Overview of the Smith build process ------------------------------------- - -The Smith build process has been broken into three phases with various related options: - -1. (Optional) Build the developer tools -2. Build the third party libraries -3. Build the Smith source code - -The developer tools are only required if you wish to contribute to the Smith source code. The first two steps involve building all of the -third party libraries that are required by Smith. Two options exist for this process: using the `Spack HPC package manager `_ -via the `uberenv wrapper script `_ or building the required dependencies on your own. We recommend the first -option as building HPC libraries by hand can be a tedious process. Once the third party libraries are built, Smith can be built using the -cmake-based `BLT HPC build system `_. - -.. _devtools-label: - -Building Smith's Developer Tools --------------------------------- - -.. note:: - This can be skipped if you are not doing Smith development or if you are on an LC machine. - They are installed in a group space defined in ``host-config/--.cmake`` - -Smith developers utilize some industry standard development tools in their everyday work. We build -these with Spack and have them installed in a public space on commonly used LC machines. These are -defined in the host-configs in our repository. - -If you wish to build them yourself (which takes a long time), use one of the following commands: - -For LC machines: - -.. code-block:: bash - - $ python3 scripts/llnl/build_devtools.py --directory= - -For other machines: - -.. code-block:: bash - - $ python3 scripts/uberenv/uberenv.py --project-json=scripts/spack/devtools.json --spack-env-file= --prefix= - -For example on **Ubuntu 24.04**: - -.. code-block:: bash - - python3 scripts/uberenv/uberenv.py --project-json=scripts/spack/devtools.json --spack-env-file=scripts/spack/configs/linux_ubuntu_24/spack.yaml --prefix=../path/to/install - -Unlike Smith's library dependencies, our developer tools can be built with any compiler because -they are not linked into the smith executable. We recommend Clang 19 because we have tested that they all -build with that compiler. - -Building Smith's Dependencies via Spack/uberenv ------------------------------------------------ - -For detailed instructions see :ref:`tpl_builds-label`. - -.. note:: - This is optional if you are on an LC machine and are in the ``smithdev`` group as we have - previously built the dependencies. You can see these machines and configurations in the - ``host-configs`` repository directory. - -Using a Docker Image with Preinstalled Dependencies ---------------------------------------------------- - -As an alternative, you can build Smith using preinstalled dependencies inside a Docker -container. Instructions for this process are located :ref:`here `. - -.. _build-label: - -Building Smith --------------- - -Smith uses a CMake build system that wraps its configure step with a script -called ``config-build.py``. This script creates a build directory and -runs the necessary CMake command for you. You just need to point the script -at the generated or a provided host-config. This can be accomplished with -one of the following commands: - -.. code-block:: bash - - # If you built Smith's dependencies yourself either via Spack or by hand - $ python3 ./config-build.py -hc .cmake - - # If you are on an LC machine and want to use our public pre-built dependencies - $ python3 ./config-build.py -hc host-configs/--.cmake - - # If you'd like to configure specific build options, e.g., a debug build - $ python3 ./config-build.py -hc /path/to/host-config.cmake -DCMAKE_BUILD_TYPE=Debug - -If you built the dependencies using Spack/uberenv, the host-config file is output at the -project root. To use the pre-built dependencies on LC, you must be in the appropriate -LC group. Contact `Brandon Talamini `_ for access. - -Some build options frequently used by Smith include: - -* ``CMAKE_BUILD_TYPE``: Specifies the build type, see the `CMake docs `_ -* ``ENABLE_BENCHMARKS``: Enables Google Benchmark performance tests, defaults to ``OFF`` -* ``ENABLE_WARNINGS_AS_ERRORS``: Turns compiler warnings into errors, defaults to ``ON`` -* ``ENABLE_ASAN``: Enables the Address Sanitizer for memory safety inspections, defaults to ``OFF`` -* ``SMITH_ENABLE_TESTS``: Enables Smith unit tests, defaults to ``ON`` -* ``SMITH_ENABLE_CODEVELOP``: Enables local development build of MFEM/Axom, see :ref:`codevelop-label`, defaults to ``OFF`` -* ``SMITH_USE_VDIM_ORDERING``: Sets the vector ordering to be ``byVDIM``, which is significantly faster for algebraic multigrid, defaults to ``ON``. - -Once the build has been configured, Smith can be built with the following commands: - -.. code-block:: bash - - $ cd build- - $ make -j16 - -.. note:: - On LC machines, it is good practice to do the build step in parallel on a compute node. - Here is an example command: ``srun -ppdebug -N1 --exclusive make -j16`` - -We provide the following useful build targets that can be run from the build directory: - -* ``test``: Runs our unit tests -* ``docs``: Builds our documentation to the following locations: - - * Sphinx: ``build-*/src/docs/html/index.html`` - * Doxygen: ``/build-*/src/docs/html/doxygen/html/index.html`` - -* ``style``: Runs styling over source code and replaces files in place -* ``check``: Runs a set of code checks over source code (CppCheck and clang-format)