A flexible package manager that supports multiple versions, configurations, platforms, and compilers.
spack external find (#31144,#31201,#31173.#31186)spack containerize (#29741,#31321)Full Changelog: https://github.com/spack/spack/compare/v0.18.0...v0.18.1
spack stage with custom paths (#30448)spack buildcache save-specfile (#30637)Full Changelog: https://github.com/spack/spack/compare/v0.17.2...v0.17.3
v0.18.0 is a major feature release.
Concretizer now reuses by default
spack install --reuse was introduced in v0.17.0, and --reuse
is now the default concretization mode. Spack will try hard to
resolve dependencies using installed packages or binaries (#30396).
To avoid reuse and to use the latest package configurations, (the
old default), you can use spack install --fresh, or add
configuration like this to your environment or concretizer.yaml:
concretizer:
reuse: false
Finer-grained hashes
Spack hashes now include link, run, and build dependencies,
as well as a canonical hash of package recipes. Previously, hashes
only included link and run dependencies (though build
dependencies were stored by environments). We coarsened the hash to
reduce churn in user installations, but the new default concretizer
behavior mitigates this concern and gets us reuse and provenance.
You will be able to see the build dependencies of new installations
with spack find. Old installations will not change and their
hashes will not be affected. (#28156, #28504, #30717, #30861)
Improved error messages
Error handling with the new concretizer is now done with optimization criteria rather than with unsatisfiable cores, and Spack reports many more details about conflicting constraints. (#30669)
Unify environments when possible
Environments have thus far supported concretization: together or
concretization: separately. These have been replaced by a new
preference in concretizer.yaml:
concretizer:
unify: [true|false|when_possible]
concretizer:unify:when_possible will try to resolve a fully
unified environment, but if it cannot, it will create multiple
configurations of some packages where it has to. For large
environments that previously had to be concretized separately, this
can result in a huge speedup (40-50x). (#28941)
Automatically find externals on Cray machines
Spack can now automatically discover installed packages in the Cray
Programming Environment by running spack external find (or spack external read-cray-manifest to only query the PE). Packages from
the PE (e.g., cray-mpich are added to the database with full
dependency information, and compilers from the PE are added to
compilers.yaml. Available with the June 2022 release of the Cray
Programming Environment. (#24894, #30428)
New binary format and hardened signing
Spack now has an updated binary format, with improvements for
security. The new format has a detached signature file, and Spack
verifies the signature before untarring or decompressing the binary
package. The previous format embedded the signature in a tar
file, which required the client to run tar before verifying
(#30750). Spack can still install from build caches using the old
format, but we encourage users to switch to the new format going
forward.
Production GitLab pipelines have been hardened to securely sign binaries. There is now a separate signing stage so that signing keys are never exposed to build system code, and signing keys are ephemeral and only live as long as the signing pipeline stage. (#30753)
Bootstrap mirror generation
The spack bootstrap mirror command can automatically create a
mirror for bootstrapping the concretizer and other needed
dependencies in an air-gapped environment. (#28556)
Initial Windows support
Spack now has initial support for Windows. Spack core has been refactored to run in the Windows environment, and a small number of packages can now build for Windows. More details are in the documentation (#27021, #28385, many more)
Makefile generation
spack env depfile can be used to generate a Makefile from an
environment, which can be used to build packages the environment
in parallel on a single node. e.g.:
spack -e myenv env depfile > Makefile
make
Spack propagates gmake jobserver information to builds so that
their jobs can share cores. (#30039, #30254, #30302, #30526)
New variant features
In addition to being conditional themselves, variants can now have conditional values that are only possible for certain configurations of a package. (#29530)
Variants can be
declared "sticky",
which prevents them from being enabled or disabled by the
concretizer. Sticky variants must be set explicitly by users
on the command line or in packages.yaml. (#28630)
run dependencies
with link:run (#29336)spack external find --all finds library-only packages in
addition to build dependencies (#28005)config:license_dir option (#30135)spack external find --path PATH takes a custom search path (#30479)spack spec has a new --format argument like spack find (#27908)spack concretize --quiet skips printing concretized specs (#30272)spack info now has cleaner output and displays test info (#22097)/hash syntax to refer to concrete specs in an environment
now works even if /hash is not installed. (#30276)0.19.0-dev0
unify: when_possible is
much faster than concretizing separately (#28941, see above)oneapi and dpcpp flag support (#30783)M1 and a64fx (#30683)Spack no longer supports Python 2.6 (#27256)
Removed deprecated --run-tests option of spack install;
use spack test (#30461)
Removed deprecated spack flake8; use spack style (#27290)
Deprecate spack:concretization config option; use
concretizer:unify (#30038)
Deprecate top-level module configuration; use module sets (#28659)
spack activate and spack deactivate are deprecated in favor of
environments; will be removed in 0.19.0 (#29430; see also link:run
in #29336 above)
--reuse (#30357, #30092, #29835, #29933, #28605, #29694, #28848)CMakePackage uses CMAKE_INSTALL_RPATH_USE_LINK_PATH (#29703)lua support: lua-lang virtual supports both
lua and luajit via new LuaPackage build system(#28854)pip (#27798)find_libraries: search for both .so and .dylib on macOS (#28924)?full_index=1 for all github patches (#29239)v0.17.0
--enable-locks behavior (#24675)develop (#27472)spack audit: fix API calls to variants (#27713)MANPATH can use system defaults (#21682)setdefault subcommand to spack module tcl (#14686)v0.17.0 is a major feature release.
New concretizer is now default
The new concretizer introduced as an experimental feature in v0.16.0
is now the default (#25502). The new concretizer is based on the
clingo logic programming system,
and it enables us to do much higher quality and faster dependency solving
The old concretizer is still available via the concretizer: original
setting, but it is deprecated and will be removed in v0.18.0.
Binary Bootstrapping
To make it easier to use the new concretizer and binary packages,
Spack now bootstraps clingo and GnuPG from public binaries. If it
is not able to bootstrap them from binaries, it installs them from
source code. With these changes, you should still be able to clone Spack
and start using it almost immediately. (#21446, #22354, #22489, #22606,
#22720, #22720, #23677, #23946, #24003, #25138, #25607, #25964, #26029,
#26399, #26599).
Reuse existing packages (experimental)
The most wanted feature from our
2020 user survey and
the most wanted Spack feature of all time (#25310). spack install,
spack spec, and spack concretize now have a --reuse option, which
causes Spack to minimize the number of rebuilds it does. The --reuse
option will try to find existing installations and binary packages locally
and in registered mirrors, and will prefer to use them over building new
versions. This will allow users to build from source far less than in
prior versions of Spack. This feature will continue to be improved, with
configuration options and better CLI expected in v0.17.1. It will become
the default concretization mode in v0.18.0.
Better error messages We have improved the error messages generated by the new concretizer by using unsatisfiable cores. Spack will now print a summary of the types of constraints that were violated to make a spec unsatisfiable (#26719).
Conditional variants
Variants can now have a when="<spec>" clause, allowing them to be
conditional based on the version or other attributes of a package (#24858).
Git commit versions
In an environment and on the command-line, you can now provide a full,
40-character git commit as a version for any package with a top-level
git URL. e.g., spack install hdf5@45bb27f58240a8da7ebb4efc821a1a964d7712a8.
Spack will compare the commit to tags in the git repository to understand
what versions it is ahead of or behind.
Override local config and cache directories
You can now set SPACK_DISABLE_LOCAL_CONFIG to disable the ~/.spack and
/etc/spack configuration scopes. SPACK_USER_CACHE_PATH allows you to
move caches out of ~/.spack, as well (#27022, #26735). This addresses
common problems where users could not isolate CI environments from local
configuration.
Improvements to Spack Containerize
For added reproducibility, you can now pin the Spack version used by
spack containerize (#21910). The container build will only build
with the Spack version pinned at build recipe creation instead of the
latest Spack version.
New commands for dealing with tags
The spack tags command allows you to list tags on packages (#26136), and you
can list tests and filter tags with spack test list (#26842).
spack diff command can diff two installed specs (#22283, #25169)spack -c <config> can set one-off config parameters on CLI (#22251)spack load --list is an alias for spack find --loaded (#27184)spack gpg can export private key with --secret (#22557)spack style automatically bootstraps dependencies (#24819)spack style --fix automatically invokes isort (#24071)--include-build-deps (#19955)spack audit command for checking package constraints (#23053)CVS repositories (yep, really) (#23212)spack monitor lets you upload analysis about installations to a
spack monitor server (#23804, #24321,
#23777, #25928))spack python --path shows which python Spack is using (#22006)spack env activate --temp can create temporary environments (#25388)--preferred and --latest options for spack checksum (#25830)cc is now pure posix and runs on Alpine (#26259)SPACK_PYTHON environment variable sets which python spack uses (#21222)SPACK_SKIP_MODULES lets you source setup-env.sh faster if you don't need modules (#24545)spec.yaml files are now spec.json, yielding a large speed improvement (#22845)x86_64_v2, x86_64_v3, x86_64_v4 targets
(see archspec#31)spack arch --generic lets you get the best generic architecture for
your node (#27061)arm compiler on graviton2 (#24904)
and on a64fx (#24524),spackbot to help package maintainers with notifications. See
spack.github.io/spackbot
spack ci rebuild (#22887)--no-add installs only specified specs and only if already present in… (#22657)spack buildcache sync command (#25470)develop and it will be unsupported in 0.18.spack setup was deprecated in v0.16.0, and has now been removed.
Use spack develop and spack dev-build.--dependencies flag from spack load (#25731)spack module [refresh|find|rm|loads], all of which
were deprecated in 2018.spack install (#21319, #27012, #25314)spack config edit now works even if spack.yaml is broken (#24689)1.1.0:1.1 (#26402).99's from many version ranges (#26422)CachedCMakePackage for using *.cmake initial config files (#19316)lua-lang allows swapping lua and luajit (#22492)ld.gold and ld.lld (#25626)$prefix/.spack (#21179)pypi attribute to infer homepage/url/list_url (#17587)config.guess file replacement (#26035)v0.16.0
spack test usage in packages is increasing
spack load and other commands. (#23661)spack fetch is now environment-aware. (#19166)clingo-based concretizer. (#23016, #23307,
#23090, #22896, #22534, #20644, #20537, #21148)clingo from source. (#20652, #20657
#21364, #21446, #21913, #22354, #22444, #22460, #22489, #22610, #22631)collections.abc (#20441)__import__ instead of Spack package import. (#23288, #23290)--source-dir argument for spack location. (#22755, #22348, #22321)spack build-env now prefers specs defined in the active environment. (#21642)from_sourcing_files. (#22767)InternalConfigScope. (#22609)SingleFileScope able to repopulate the cache after clearing it. (#22559)package.py (#21811)autogen.sh. (#20319)-k/verify-ssl-false in _existing_url method. (#21864)This minor release includes a new feature and associated fixes:
This release also contains bug fixes/enhancements for:
plus assorted documentation (#20021, #20174) and package bug fixes/enhancements (#19617, #19933, #19986, #20006, #20097, #20198, #20794, #20906, #21411).
v0.16.0 is a major feature release.
New concretizer (experimental) Our new backtracking concretizer is
now in Spack as an experimental feature. You will need to install
clingo@master+python and set concretizer: clingo in config.yaml
to use it. The original concretizer is not exhaustive and is not
guaranteed to find a solution if one exists. We encourage you to use
the new concretizer and to report any bugs you find with it. We
anticipate making the new concretizer the default and including all
required dependencies for it in Spack v0.17. For more details, see
#19501.
spack test (experimental) Users can add test() methods to their
packages to run smoke tests on installations with the new spack test
command (the old spack test is now spack unit-test). spack test
is environment-aware, so you can spack install an environment and
spack test run smoke tests on all of its packages. Historical test
logs can be perused with spack test results. Generic smoke tests for
MPI implementations, C, C++, and Fortran compilers as well as specific
smoke tests for 18 packages. This is marked experimental because the
test API (self.run_test()) is likely to be change, but we encourage
users to upstream tests, and we will maintain and refactor any that
are added to mainline packages (#15702).
spack develop New spack develop command allows you to develop
several packages at once within a Spack environment. Running
spack develop foo@v1 and spack develop bar@v2 will check
out specific versions of foo and bar into subdirectories, which you
can then build incrementally with spack install (#15256).
More parallelism Spack previously installed the dependencies of a single spec in parallel. Entire environments can now be installed in parallel, greatly accelerating builds of large environments. get parallelism from individual specs. Spack now parallelizes entire environment builds (#18131).
Customizable base images for spack containerize
spack containerize previously only output a Dockerfile based
on official Spack images from Dockerhub. You may now specify
any base image of your choosing (#15028).
more external finding spack external find was added in v0.15,
but only cmake had support. spack external find can now find
bison, cuda, findutils, flex, git, lustre m4, mpich,
mvapich2, ncurses, openmpi, perl, spectrum-mpi, tar, and
texinfo on your system and add them automatically to
packages.yaml.
Support aocc, nvhpc, and oneapi compilers We are aggressively
pursuing support for the newest vendor compilers, especially those for
the U.S. exascale and pre-exascale systems. Compiler classes and
auto-detection for aocc, nvhpc, oneapi are now in Spack (#19345,
#19294, #19330).
spack mark command can be used to designate packages as explicitly
installed, so that spack gc will not garbage-collect them (#16662).install_tree can be customized with Spack's projection format (#18341)sbang now lives in the install_tree so that all users can access it (#11598)csh and tcsh users no longer need to set SPACK_ROOT before
sourcing setup-env.csh (#18225)variant=* syntax for finding any package
that has a particular variant (#19381).SPACK_GNUPGHOME variable for custom GPG directories (#17139)sbang is an external package now (https://github.com/spack/sbang, #19582)archspec is an external package now (https://github.com/archspec/archspec, #19600)spack bootstrap was deprecated in v0.14.0, and has now been removed.spack setup is deprecated as of v0.16.0.spack test is now called spack unit-test. spack test is
now the smoke testing feature in (2) above.Some of the most notable bugfixes in this release include:
packages.yaml (#18013)buildcache list --allarch now works properly (#17827)Spack now has 5050 total packages, 720 of which were added since v0.15.
hip, aomp, more) added by AMD (#19957, #19832, others)llvm-flang, flang, and f18 removed, as llvm has real flang
support since Flang was merged to LLVM mainlinespack external find and spack test in packages.gitlab.spack.io
develop