Building Spheral

This guide is designed to help users build and install Spheral external (meaning non-Livermore Computing systems). This guide assumes the user is on a Linux based system.

System Requirements

Spheral has system requirements and recommendations. Many come installed on Linux distributions by default. The package names will vary depending on what distribution/version you have. Spheral uses Spack under the hood to handle building our Third Party Library (TPL) dependencies. Users are not required to install Spack themselves, but they can use the System Prerequisites page for Spack to check the proper system requirements.

Note

Any packages that Spack does not find on the system will be installed through Spack, potentially causing excessively long install times. To avoid this, there are some packages we require (and some we recommend) installing for the system first.

Warning

These instructions are currently not applicable to newer distributions that ship with Python 3.13 (like Fedora 42). We hope to remedy this in the near future.

Select the following dropdown menu for the appropriate commands to run for a given Linux distribution and version.

Ubuntu

Version 20.04

# Required packages
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bzip2 ca-certificates g++ gcc gfortran git gzip
sudo apt-get install lsb-release patch python3 tar unzip xz-utils zstd
sudo apt-get install libtool curl wget libcurl4-openssl-dev tk-dev autotools-dev
sudo apt-get install build-essential python3-dev python3-pip python3-venv

# Recommended packages
sudo apt-get install cmake autoconf automake mpich libreadline-dev

Version 22.04/24.04

# Required packages
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bzip2 ca-certificates g++ gcc gfortran git gzip
sudo apt-get install lsb-release patch python3 tar unzip xz-utils zstd
sudo apt-get install libtool curl wget libcurl4-openssl-dev tk-dev autotools-dev
sudo apt-get install build-essential python3-dev python3-pip python3-venv

# Recommended packages (MPICH library is broken for 22.04/24.04, use openmpi)
sudo apt-get install cmake autoconf automake libopenmpi-dev libreadline-dev

RHEL/AlmaLinux

Version 8

# Required packages
dnf update
dnf install epel-release
dnf group install "Development Tools"
dnf install gcc-fortran gcc-c++ redhat-lsb-core unzip python3-devel

# Recommended packages
dnf install environment-modules cmake autoconf automake mpich-devel ncurses
# Be sure to add your mpi install to your PATH so Spack can find it
module load mpi

Version 9

# Required packages
dnf update
dnf install epel-release
dnf group install "Development Tools"
dnf install gcc-fortran gcc-c++ unzip python3-devel

# Recommended packages
dnf install environment-modules cmake autoconf automake mpich-devel ncurses
# Be sure to add your mpi install to your PATH so Spack can find it
module load mpi

Cloning/Updating

Spheral uses multiple submodules, including BLT and PYB11Generator. Because of this, be sure to use the --recursive option when cloning Spheral:

git clone --recursive https://github.com/LLNL/Spheral

If you forget to use the --recursive argument or if you checkout from a different branch, you must run:

git submodule update --init --recursive

Similarly, to update an existing Spheral repo, use

git pull
git submodule sync --recursive
git submodule update --init --recursive

Third Party Libraries (TPLs)

Spheral uses Spack to facilitate building and linking in external Third Party Libraries (TPLs).

Running TPL Manager

Spheral provides a tool (tpl-manager.py) in an attempt to simplify the Spack process for the user, which does the following:

• Clone and setup a local Spack instance for Spheral.

• Activate (and create, if necessary) a Spack environment for the current system architecture.

• Generate a dependency tree of third party libraries (relative to the provided configuration).

• Build and install all dependent libraries in the local Spack instance.

• Generate a CMake host-config file for configuring Spheral builds. This file will be named <sys_type>-<spec>.cmake.

tpl-manager.py is located in scripts/devtools and must be run from the Spheral root directory. The most common usage of tpl-manager.py will look like:

./scripts/devtools/tpl-manager.py --spec spheral+mpi%gcc

The command above tells the TPL manager to build and install TPLs for Spheral with MPI enabled and using the GCC compiler. Overall, the following options can be provided to tpl-manager.py:

--spec <SPEC>

Spheral Spack spec to build. Optional, default behavior is to build all spec permutations in the current environment. Required if running on non-LC systems. See the Spack documentation on specs-dependencies for more information.

--show-specs

Display specs found in current environment, if environment exists.

--show-info

Display Spheral Spack info, including variants and dependencies.

--spack-dir <SPACK_DIR>

Where to create the Spack instance to use. Optional, default ./../spheral-spack-tpls.

--clean

Cleans Spack repo instance and forces Spack to reconcretize environment. Use if running into odd behavior with the script.

More tpl-manager.py details:

When building TPLs on non-LC systems, tpl-manager.py will :

1. Create and activate a Spack environment in spheral/scripts/spack/environments based on the output of spack arch. All subsequent Spack commands will modify this environment by adding specs, compilers, and external packages to it.

2. Run spack compiler find to find system compilers. Spack searches the $PATH environment variable for compilers and packages. Add any paths to $PATH to ensure Spack will find them before running tpl-manager.py.

3. Run spack external find to try to find existing system installs for things like CMake, Git, Python, and MPICH/OpenMPI (when the spec has +mpi). Be sure to add any packages to your $PATH before running tpl-manager.py. This can involve using module load <module name> or similar methods.

4. Add the current spec to the environment with spack add <spec> and concretize using spack concretize -U.

5. Install the TPLs and create the host config file for the given spec with spack install -u initconfig <spec>. The script will print the name of the host config file it created to the terminal. The form will be of the form <sys-arch>-<compiler>.cmake.

Configuring

After running tpl-manager, you will see a file in your Spheral root directory following the format <sys_type>-<spec>.cmake. This is the host config file that will be used to configure a new build of Spheral. We provide a tool at scripts/devtools/host-config-build.py to simplify Spheral’s CMake setup. Use it with the host config file like so:

./scripts/devtools/host-config-build.py --host-config <sys_type>-<spec>.cmake

Running host-config-build.py

Overall, the following options can be provided to host-config-build.py

--host-config <HOST_CONFIG_FILE>

Name of the host config file created by tpl-manager.py. Typically named <sys_type>-<spec>.cmake. Required.

-s, --source-dir <SOURCE_DIR>

Location of a Spheral source repo. Optional, default is current directory.

--build-dir <BUILD_DIR>

Spheral build directory. Optional, default <SOURCE_DIR>/build_<sys-type>-<spec>. Makes the Spheral build directory in <BUILD_DIR>/build.

-i, --install-dir <INSTALL_DIR>

Spheral installation directory. Optional, default <SOURCE_DIR>/<BUILD_DIR>/install.

--no-clean

Do not delete build and install locations. Optional.

--build

Run make -j <N> install after configuring. Optional.

-D<CMAKE_ARGUMENTS>

Any additional CMake arguments. See Spheral CMake Configurations for possible options.

Example:

./scripts/devtools/host-config-build.py --host-config linux-ubuntu20-gcc@10.3.1.cmake -DENABLE_DEV_BUILD=ON

Build and Install

In order to build Spheral, navigate to the <BUILD_DIR> from the Configure step and run make -j <N> install, where <N> should be replaced with the number of processes to use (typicall the number of cores on your CPU).

Although Spheral is simply a set of Python modules, Spheral sets up a Python virtual environment during the install stage. The install will also download and install all required runtime python libraries needed to execute the full Spheral test suite.

Users will find a spheral script in the install directory, which can be run as ./spheral. This will drop users into the active Spheral virtual environment, providing access to all Spheral libraries and dependencies. To ensure Spheral uses multiple ranks, run ./spheral with the appropriate run command, i.e. srun -n <number of procs> ./spheral for Slurm machines, flux run -xN <number of nodes> ./spheral, etc.

Running Tests

Basic Smoke Test

After a build and install, we recommend you perform a smoke test with Spheral to ensure the Spheral environment is installed and linked correctly.

From your install directory run:

./spheral -c "import Spheral"

This will run the Spheral python process and import the Spheral modules. If successful, you should see a banner printed with information about the version of Spheral, after which Python exits.

ATS Testing Suite

Spheral uses ATS (Automated Testing System) to execute a suite of parallel tests. During install, a script spheral-ats is created. This script handles launching allocations and setting appropriate ATS and hardware flags for running the testing suite.

From the install directory run:

./spheral-ats tests/integration.ats

For more information about using spheral-ats, run the following from the install directory:

./spheral-ats --help