...
Versions installed in Pawsey systems
To check the current installed versions, use the module avail command (current versions may be different from content shown here):
| Column |
|---|
|
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 1. Checking for installed versions |
|---|
| $ module avail python
--------------------- /software/setonix/2024.05/modules/zen3/gcc/12.2.0/programming-languages ---------------------
python/3.9.15 python/3.11.6 (D)
--------------------------------------- /opt/cray/pe/lmod/modulefiles/core ----------------------------------------
cray-python/3.9.13.1 cray-python/3.10.10 (D) |
|
We recommend the use of python installed by Pawsey instead of cray-python.
Many scientific and managing tools have been also installed in our software stack. These are named with the prefix py-, and sometimes their flavour is also identified by a suffix -py<VERSION> after the version corresponding to the tool itself (current versions may be different from content shown here):
| Column |
|---|
|
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 1. Checking for installed versions |
|---|
| $ module avail py-
---------------------- /software/setonix/2024.05/modules/zen3/gcc/12.2.0/astro-applications -----------------------
py-astropy/4.2.1 py-astropy/5.1 (D) py-emcee/3.1.1 py-funcsigs/1.0.2
--------------------------- /software/setonix/2024.05/modules/zen3/gcc/12.2.0/utilities ---------------------------
py-boto3/1.26.26 py-setuptools/59.4.0-py3.9.15 py-setuptools/68.0.0-py3.11.6 (D)
py-pip/23.1.2-py3.9.15 py-setuptools/59.4.0-py3.11.6
py-pip/23.1.2-py3.11.6 (D) py-setuptools/68.0.0-py3.9.15
------------------------ /software/setonix/2024.05/modules/zen3/gcc/12.2.0/python-packages ------------------------
py-cython/0.29.36 py-ipython/8.14.0 py-numpy/1.24.4 py-scikit-learn/1.3.2
py-cython/3.0.4 (D) py-matplotlib/3.8.1 py-numpy/1.26.1 (D) py-scipy/1.11.3
py-dask/2023.4.1 py-mpi4py/3.1.5-py3.11.6 py-pandas/1.5.3
py-h5netcdf/0.10.0 py-netcdf4/1.6.2 py-pandas/2.1.2 (D)
py-h5py/3.8.0 py-numba/0.57.0 py-plotly/5.14.1
------------------------ /software/setonix/2024.05/modules/zen3/gcc/12.2.0/developer-tools ------------------------
py-hatchet/1.3.1 |
|
Before you begin
Basic information on these installation tools can be found in their websites (external links):
...
Pawsey Python modules also preconfigure the shell environment by default to provide a meaningful location for user-specific package installations:
...
| Column |
|---|
|
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 1: Set up environment for pip |
|---|
| $ module load python/3.11.6<VERSION_OF_PYTHON>
$ module load py-pip/23.1.2-py3.11.6<VERSION>-py<VERSION_OF_PYTHON>
$ module load py-numpy/1.26.1<VERSION> |
|
Now let's proceed with the installation. We're going to use pip with the --user flag to install in the Python user directory. (Users cannot install software in system directories on shared Pawsey supercomputer systems.)
...
| Column |
|---|
|
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 4: Pip install from requirements |
|---|
| $ module load python/3.9.7<VERSION_OF_PYTHON>
$ module load py-pip/21.1.2-py3.9.7<VERSION>-py<VERSION_OF_PYTHON>
$ module load py-numpy/1.20.3<VERSION>
$ sg <projectcode> -c 'pip install --user --no-deps -r requirements.txt'
Collecting astropy==4.1 (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/74/9c/a1e51955d4a2af497a507c323409ebe55c122a91c438d2884d918360efc1/astropy-4.1-cp36-cp36m-manylinux1_x86_64.whl (10.3MB)
|████████████████████████████████| 10.3MB 16.0MB/s
Requirement already satisfied: numpy==1.19.0 in /pawsey/sles12sp3/python/3.6.3/numpy/1.19.0/lib/python3.6/site-packages/numpy-1.19.0-py3.6-linux-x86_64.egg (from -r requirements.txt (line 2)) (1.19.0)
Installing collected packages: astropy
Successfully installed astropy-4.1
|
|
...
Change to the tqdm source code directory and then load the required modules, namely python and py-setuptools:
$ module load python/3.9.7 <VERSION_OF_PYTHON> py-setuptools/57.4.0-py3.9.7<VERSION>-py<VERSION_OF_PYTHON>
Finally, execute the build and installation process:
...