Lmod modules¶
Lmod is a Lua-based environment module system used to dynamically
modify the shell environment. Modules allow users to easily configure
their software environment by adjusting variables such as PATH,
LD_LIBRARY_PATH, and other application-specific settings.
Using modules simplifies switching between different software versions and toolchains without modifying system configuration.
Detailed documentation on Lmod can be found in the Lmod documentation.
Useful Commands¶
You can quickly set up your environment using modules; the following commands are useful:
module avail # list all available modules
module load <module_name> # load desired module
module list # list all currently loaded modules
module unload <module_name> # unload desired module
module purge # unload all modules
module use <path_to_modules> # use the modules stored in the <path_to_modules> folder
module help # display general help
module help <module_name> # display information about module
module keyword <key> # list modules matching the <key> keyword
module spider <key> # similar to keyword command, more effective and precise
The command ml is a shorter alias for module and can be used in the
same way.
Listing Loaded Modules¶
To get an overview of the currently loaded modules, use module list or ml:
ml
Currently Loaded Modules:
1) EasyBuild/4.7.2
Searching for Available Modules¶
To get an overview of all available modules, you can use module avail or simply ml av:
Available modules on Devana
login01:~ $ module avail
R/4.2.3 (D) libvirt/8.10.0 notebook/6.4.10 nvhpc-nompi/23.1 qemu/6.2.0
cuda/12.0.1 maqao/2.17.4 nvhpc-byo-compiler/23.1 nvhpc/23.1 singularity/ce-3.11.0
gcc/12.2 nmap/7.93 nvhpc-hpcx/23.1 openmpi/4.1.4/gcc-12.2 websockify/0.10.0
... ... ...
---------------------------------------- /storage-apps/intel-v.2023/oneapi/modulefiles ----------------------------------------
advisor/latest debugger/2023.0.0 (D) icc32/latest mkl/2023.0.0 (D)
advisor/2023.0.0 (D) dev-utilities/latest icc32/2023.0.0 (D) mkl32/latest
ccl/latest dev-utilities/2021.8.0 (D) init_opencl/latest mkl32/2023.0.0 (D)
ccl/2021.8.0 (D) dnnl-cpu-gomp/latest init_opencl/2023.0.0 (D) mpi/latest
clck/latest dnnl-cpu-gomp/2023.0.0 (D) inspector/latest mpi/2021.8.0 (D)
clck/2021.7.2 (D) dnnl-cpu-iomp/latest inspector/2023.0.0 (D) oclfpga/latest
compiler-rt/latest dnnl-cpu-iomp/2023.0.0 (D) intel_ipp_ia32/latest oclfpga/2023.0.0 (D)
compiler-rt/2023.0.0 (D) dnnl-cpu-tbb/latest intel_ipp_ia32/2021.7.0 (D) tbb/latest
compiler-rt32/latest dnnl-cpu-tbb/2023.0.0 (D) intel_ipp_intel64/latest tbb/2021.8.0 (D)
compiler-rt32/2023.0.0 (D) dnnl/latest intel_ipp_intel64/2021.7.0 (D) tbb32/latest
compiler/latest dnnl/2023.0.0 (D) intel_ippcp_ia32/latest tbb32/2021.8.0 (D)
compiler/2023.0.0 (D) dpct/latest intel_ippcp_ia32/2021.6.3 (D) vpl/latest
compiler32/latest dpct/2023.0.0 (D) intel_ippcp_intel64/latest vpl/2023.0.0 (D)
compiler32/2023.0.0 (D) dpl/latest intel_ippcp_intel64/2021.6.3 (D) vtune/latest
dal/latest dpl/2022.0.0 (D) itac/latest vtune/2023.0.0 (D)
dal/2023.0.0 (D) icc/latest itac/2021.8.0 (D)
debugger/latest icc/2023.0.0 (D) mkl/latest
... ... ...
The string before the / symbol corresponds to the software name, while the string after the / symbol corresponds to the software version and/or the compiler toolchain that was used to install the software.
Defaults
(D) indicates the default version of the software that will be
loaded if no version/toolchain is specified in the module load
command. It is recommended to always use the full module name, as
the default values can and will change upon installation of newer
versions.
Exploring Modules¶
The module spider command provides more detailed search results and
helps locate modules in the hierarchical module tree.
Module exploration
login01:~ $ module spider Python
Description:
Python is a programming language that lets you work more quickly and integrate your systems more effectively.
Versions:
Python/2.7.18-GCCcore-10.3.0-bare
Python/2.7.18-GCCcore-11.3.0-bare
Python/3.7.4-GCCcore-8.3.0
Python/3.8.6-GCCcore-10.2.0
Python/3.9.5-GCCcore-10.3.0-bare
Python/3.9.5-GCCcore-10.3.0
Python/3.9.6-GCCcore-11.2.0-bare
Python/3.9.6-GCCcore-11.2.0
Python/3.10.4-GCCcore-11.3.0-bare
Python/3.10.4-GCCcore-11.3.0
Other possible modules matches:
python
We can now choose the desired version of Python, e.g. Python/3.10.4-GCCcore-11.3.0:
module load Python/3.10.4-GCCcore-11.3.0
You can also use module commands in your Slurm batch scripts.
Inspecting a Module¶
To see how a module would change the environment, use ml show:
Inspecting module
ml show Python/3.10.4-GCCcore-11.3.0
---------------------------------------------------------------------------------------------------------------------------
/storage-apps/easybuild/modules/lang/Python/3.10.4-GCCcore-11.3.0.lua:
---------------------------------------------------------------------------------------------------------------------------
help([[
Description
===========
Python is a programming language that lets you work more quickly and integrate your systems
more effectively.
More information
================
- Homepage: https://python.org/
Included extensions
===================
alabaster-0.7.12, appdirs-1.4.4, asn1crypto-1.5.1, atomicwrites-1.4.0,
attrs-21.4.0, Babel-2.10.1, backports.entry_points_selectable-1.1.1,
backports.functools_lru_cache-1.6.4, bcrypt-3.2.2, bitstring-3.1.9,
...
webencodings-0.5.1, wheel-0.37.1, xlrd-2.0.1, zipfile36-0.1.3, zipp-3.8.0
]])
whatis("Description: Python is a programming language that lets you work more quickly and integrate your systems more effectively.")
whatis("Homepage: https://python.org/")
whatis("URL: https://python.org/")
whatis("Extensions: alabaster-0.7.12, appdirs-1.4.4, asn1crypto-1.5.1, atomicwrites-1.4.0, attrs-21.4.0, Babel-2.10.1, backports.entry_points_selectable-1.1.1, backports.functools_lru_cache-1.6.4, bcrypt-3.2.2, bitstring-3.1.9, blist-1.3.6, CacheControl-0.12.11, cachy-0.3.0, certifi-2021.10.8, cffi-1.15.0, chardet-4.0.0, charset-normalizer-2.0.12, cleo-0.8.1, click-8.1.3, clikit-0.6.2, colorama-0.4.4, crashtest-0.3.1, cryptography-37.0.1, Cython-0.29.28, decorator-5.1.1, distlib-0.3.4, docopt-0.6.2, docutils-0.17.1, ecdsa-0.17.0, editables-0.3, filelock-3.6.0, flit-3.7.1, flit-core-3.7.1, fsspec-2022.3.0, future-0.18.2, glob2-0.7, html5lib-1.1, idna-3.3, imagesize-1.3.0, importlib_metadata-4.11.3, importlib_resources-5.7.1, iniconfig-1.1.1, intervaltree-3.1.0, intreehooks-1.0, ipaddress-1.0.23, jeepney-0.8.0, Jinja2-3.1.2, joblib-1.1.0, jsonschema-4.4.0, keyring-23.5.0, keyrings.alt-4.1.0, liac-arff-2.5.0, lockfile-0.12.2, MarkupSafe-2.1.1, mock-4.0.3, more-itertools-8.12.0, msgpack-1.0.3, netaddr-0.8.0, netifaces-0.11.0, packaging-20.9, paramiko-2.10.4, pastel-0.2.1, pathlib2-2.3.7.post1, pathspec-0.9.0, pbr-5.8.1, pexpect-4.8.0, pip-22.0.4, pkginfo-1.8.2, platformdirs-2.4.1, pluggy-1.0.0, poetry-1.1.13, poetry-core-1.0.8, psutil-5.9.0, ptyprocess-0.7.0, py-1.11.0, py_expression_eval-0.3.14, pyasn1-0.4.8, pycparser-2.21, pycrypto-2.6.1, Pygments-2.12.0, pylev-1.4.0, PyNaCl-1.5.0, pyparsing-3.0.8, pyrsistent-0.18.1, pytest-7.1.2, python-dateutil-2.8.2, pytoml-0.1.21, pytz-2022.1, regex-2022.4.24, requests-2.27.1, requests-toolbelt-0.9.1, scandir-1.10.0, SecretStorage-3.3.2, semantic_version-2.9.0, setuptools-62.1.0, setuptools-rust-1.3.0, setuptools_scm-6.4.2, shellingham-1.4.0, simplegeneric-0.8.1, simplejson-3.17.6, six-1.16.0, snowballstemmer-2.2.0, sortedcontainers-2.4.0, Sphinx-4.5.0, sphinx-bootstrap-theme-0.8.1, sphinxcontrib-applehelp-1.0.2, sphinxcontrib-devhelp-1.0.2, sphinxcontrib-htmlhelp-2.0.0, sphinxcontrib-jsmath-1.0.1, sphinxcontrib-qthelp-1.0.3, sphinxcontrib-serializinghtml-1.1.5, sphinxcontrib-websupport-1.2.4, tabulate-0.8.9, threadpoolctl-3.1.0, toml-0.10.2, tomli-2.0.1, tomli_w-1.0.0, tomlkit-0.10.2, typing_extensions-4.2.0, ujson-5.2.0, urllib3-1.26.9, virtualenv-20.14.1, wcwidth-0.2.5, webencodings-0.5.1, wheel-0.37.1, xlrd-2.0.1, zipfile36-0.1.3, zipp-3.8.0")
conflict("Python")
load("GCCcore/11.3.0")
...
prepend_path("XDG_DATA_DIRS","/storage-apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/share")
setenv("EBROOTPYTHON","/storage-apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0")
setenv("EBVERSIONPYTHON","3.10.4")
setenv("EBDEVELPYTHON","/storage-apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/easybuild/Python-3.10.4-GCCcore-11.3.0-easybuild-devel")
prepend_path("PYTHONPATH","/storage-apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/easybuild/python")
setenv("EBEXTSLISTPYTHON","wheel-0.37.1,setuptools-62.1.0,pip-22.0.4,blist-1.3.6,pbr-5.8.1,Cython-0.29.28,six-1.16.0,toml-0.10.2, flit-core-3.7.1,tomli-2.0.1,setuptools_scm-6.4.2,python-dateutil-2.8.2,decorator-5.1.1,liac-arff-2.5.0,pycrypto-2.6.1,ecdsa-0.17.0, ipaddress-1.0.23,asn1crypto-1.5.1,idna-3.3,pycparser-2.21,cffi-1.15.0,semantic_version-2.9.0,typing_extensions-4.2.0,setuptools-rust-1.3.0, cryptography-37.0.1,pyasn1-0.4.8,PyNaCl-1.5.0,bcrypt-3.2.2,paramiko-2.10.4,pyparsing-3.0.8,netifaces-0.11.0,netaddr-0.8.0, mock-4.0.3,pytz-2022.1,bitstring-3.1.9,appdirs-1.4.4,distlib-0.3.4,filelock-3.6.0,zipp-3.8.0,importlib_metadata-4.11.3, backports.entry_points_selectable-1.1.1,pathspec-0.9.0,pluggy-1.0.0,packaging-20.9,editables-0.3,platformdirs-2.4.1,scandir-1.10.0, pathlib2-2.3.7.post1,importlib_resources-5.7.1,virtualenv-20.14.1,docopt-0.6.2,joblib-1.1.0,chardet-4.0.0,certifi-2021.10.8, urllib3-1.26.9,charset-normalizer-2.0.12,requests-2.27.1,xlrd-2.0.1,py_expression_eval-0.3.14,tabulate-0.8.9,ujson-5.2.0, atomicwrites-1.4.0,py-1.11.0,more-itertools-8.12.0,attrs-21.4.0,backports.functools_lru_cache-1.6.4,wcwidth-0.2.5,iniconfig-1.1.1, colorama-0.4.4,pytest-7.1.2,MarkupSafe-2.1.1,Jinja2-3.1.2,sphinxcontrib-serializinghtml-1.1.5,sphinxcontrib-websupport-1.2.4,Pygments-2.12.0, imagesize-1.3.0,docutils-0.17.1,snowballstemmer-2.2.0,alabaster-0.7.12,sphinxcontrib-applehelp-1.0.2,sphinxcontrib-devhelp-1.0.2, sphinxcontrib-htmlhelp-2.0.0,sphinxcontrib-jsmath-1.0.1,sphinxcontrib-qthelp-1.0.3,Babel-2.10.1,Sphinx-4.5.0,sphinx-bootstrap-theme-0.8.1, click-8.1.3,psutil-5.9.0,future-0.18.2,sortedcontainers-2.4.0,intervaltree-3.1.0,pytoml-0.1.21,zipfile36-0.1.3,tomli_w-1.0.0,flit-3.7.1, regex-2022.4.24,intreehooks-1.0,pylev-1.4.0,pastel-0.2.1,crashtest-0.3.1,clikit-0.6.2,jeepney-0.8.0,SecretStorage-3.3.2,keyring-23.5.0, keyrings.alt-4.1.0,tomlkit-0.10.2,shellingham-1.4.0,requests-toolbelt-0.9.1,pyrsistent-0.18.1,pkginfo-1.8.2,ptyprocess-0.7.0,pexpect-4.8.0, jsonschema-4.4.0,simplejson-3.17.6,webencodings-0.5.1,html5lib-1.1,cleo-0.8.1,cachy-0.3.0,msgpack-1.0.3,CacheControl-0.12.11,lockfile-0.12.2, poetry-core-1.0.8,glob2-0.7,poetry-1.1.13,fsspec-2022.3.0,threadpoolctl-3.1.0,simplegeneric-0.8.1")
Loading Modules¶
To load the desired module use module load or ml followed by the name of the module:
Loading Modules
module load Python/3.10.8-GCCcore-12.2.0
module list
Currently Loaded Modules:
1) GCCcore/12.2.0 6) libreadline/8.2-GCCcore-12.2.0 11) libffi/3.4.4-GCCcore-12.2.0
2) zlib/1.2.12-GCCcore-12.2.0 7) Tcl/8.6.12-GCCcore-12.2.0 12) OpenSSL/1.1
3) binutils/2.39-GCCcore-12.2.0 8) SQLite/3.39.4-GCCcore-12.2.0 13) Python/3.10.8-GCCcore-12.2.0
4) bzip2/1.0.8-GCCcore-12.2.0 9) XZ/5.2.7-GCCcore-12.2.0
5) ncurses/6.3-GCCcore-12.2.0 10) GMP/6.2.1-GCCcore-12.2.0
Dependencies
Note that even though we only loaded a single module,
the output of module list shows that a whole set of modules was loaded.
These are required dependencies for Python/3.10.8-GCCcore-12.2.0.
Conflicting Modules¶
It is important to note that only modules that are compatible with each other can be loaded together. In particular, modules must be installed either with the same toolchain as the modules that are already loaded, or with a compatible (sub)toolchain.
Once you have loaded one or more modules that were installed with the intel/2022b toolchain,
all other modules that you load should have been installed with the same toolchain.
Furthermore, only one single version of each software package can be loaded
at a given time. For example, once you have the Python/3.10.4-GCCcore-11.3.0
module loaded, you cannot load a different version of Python in the same
session/job script, neither directly nor indirectly as a dependency of another module you want to load.
Unloading Modules¶
To unload any module you can use ms - or ml unload followed by the name of the module:
Unloading Modules
login01:~ $ module load Python/3.10.8-GCCcore-12.2.0
login01:~ $ module list
Currently Loaded Modules:
1) GCCcore/12.2.0 4) bzip2/1.0.8-GCCcore-12.2.0 7) Tcl/8.6.12-GCCcore-12.2.0 10) GMP/6.2.1-GCCcore-12.2.0 13) Python/3.10.8-GCCcore-12.2.0
2) zlib/1.2.12-GCCcore-12.2.0 5) ncurses/6.3-GCCcore-12.2.0 8) SQLite/3.39.4-GCCcore-12.2.0 11) libffi/3.4.4-GCCcore-12.2.0
3) binutils/2.39-GCCcore-12.2.0 6) libreadline/8.2-GCCcore-12.2.0 9) XZ/5.2.7-GCCcore-12.2.0 12) OpenSSL/1.1
login01:~ $ module unload GCCcore/12.2.0
login01:~ $ module list
Currently Loaded Modules:
1) zlib/1.2.12-GCCcore-12.2.0 3) bzip2/1.0.8-GCCcore-12.2.0 5) libreadline/8.2-GCCcore-12.2.0 7) SQLite/3.39.4-GCCcore-12.2.0 9) GMP/6.2.1-GCCcore-12.2.0 11) OpenSSL/1.1
2) binutils/2.39-GCCcore-12.2.0 4) ncurses/6.3-GCCcore-12.2.0 6) Tcl/8.6.12-GCCcore-12.2.0 8) XZ/5.2.7-GCCcore-12.2.0 10) libffi/3.4.4-GCCcore-12.2.0 12) Python/3.10.8-GCCcore-12.2.0
To unload all loaded modules, simply issue module purge.
Saving module selection¶
If you have a set of modules that you load often you can store them in a collection.
To store all loaded modules in a <my_favourite_collection> collection,
issue ml save <my_favourite_collection>.
If you later wish to load these modules, simply issue ml restore <my_favourite_collection>.
The list of all your collections can be viewed with ml savelist, and
the content of a collection can be inspected with ml describe <collection>.
To remove a module collection, delete the corresponding file under $HOME/.lmod.d/collections/.