Skip to content

SDSC: Reconsider variant name scheme #165

Open

Description

@mkandes

Why is the short variant hash appended twice?

[mkandes@login01 ~]$ . /cm/shared/apps/spack/0.21.2/cpu/dev/share/spack/setup-env.sh
[mkandes@login01 ~]$ module use /cm/shared/apps/spack/0.21.2/cpu/dev/share/spack/lmod/linux-rocky8-x86_64/Core
[mkandes@login01 ~]$ module load gcc/13.3.0-t46rsdv 

Inactive Modules:
  1) slurm

[mkandes@login01 ~]$ module load openmpi/4.1.8-4ex7oqp 

Activating Modules:
  1) slurm/23.02.7-qwmhq6o-qwmhq6o

The following have been reloaded with a version change:
  1) zlib-ng/2.1.4-f3aqwo6-f3aqwo6 => zlib-ng/2.1.4-otfrjgj-otfrjgj
  2) zstd/1.5.5-a6unagk-a6unagk => zstd/1.5.5-el7dac3-el7dac3

[mkandes@login01 ~]$ module list

Currently Loaded Modules:
  1) shared
  2) cpu/0.17.3b                          (c)
  3) sdsc/1.0
  4) DefaultModules
  5) gmp/6.2.1-siwf7iu-siwf7iu
  6) mpfr/4.2.0-lp44usi-lp44usi
  7) mpc/1.3.1-sw4uskz-sw4uskz
  8) gcc/13.3.0-t46rsdv
  9) libpciaccess/0.17-mcmk6gg-mcmk6gg
 10) libiconv/1.17-sbpsu2v-sbpsu2v
 11) xz/5.4.1-3ccc2bb-3ccc2bb
 12) zlib-ng/2.1.4-otfrjgj-otfrjgj
 13) libxml2/2.10.3-u5bhlww-u5bhlww
 14) ncurses/6.4-vxmqbkn-vxmqbkn
 15) hwloc/2.9.1-raenanb-raenanb
 16) lustre/2.15.2-nhfixh7-nhfixh7
 17) numactl/2.0.14-zmmgl4f-zmmgl4f
 18) bzip2/1.0.8-2dqjb6v-2dqjb6v
 19) pigz/2.8-gnuvmhy-gnuvmhy
 20) zstd/1.5.5-el7dac3-el7dac3
 21) tar/1.34-vgozjui-vgozjui
 22) gettext/0.22.3-pnkku56-pnkku56
 23) openssl/1.1.1k-xqlkwu5-xqlkwu5
 24) krb5/1.20.1-v55lent-v55lent
 25) libedit/3.1-20210216-m76neoh-m76neoh
 26) libxcrypt/4.4.35-bnur6is-bnur6is
 27) openssh/9.5p1-bcnejst-bcnejst
 28) libevent/2.1.12-ntjypwd-ntjypwd
 29) pmix/5.0.1-ukntpod-ukntpod
 30) slurm/23.02.7-qwmhq6o-qwmhq6o
 31) knem/1.1.4-5iz5xmb-5iz5xmb
 32) rdma-core/43.0-4cn6oay-4cn6oay
 33) xpmem/2.4.6-sjtxr6s-sjtxr6s
 34) ucx/1.14.1-npuqcqj-omp
 35) openmpi/4.1.8-4ex7oqp

  Where:
   c:  built natively for AMD Rome

 

[mkandes@login01 ~]$

Activity

  1. mkandes commented on Nov 24, 2025

    @mkandes
    MemberAuthor

    No ~/.lmod.d config present ...

    [mkandes@login01 ~]$ ls -lahtr
    total 787M
    -rw-r--r--   1 mkandes use300   18 Sep  7  2020 .bash_logout
    -rw-r--r--   1 mkandes use300  334 Sep  7  2020 .emacs
    -rw-r--r--   1 mkandes use300   17 Jan  7  2021 .vimrc
    -rw-r--r--   1 mkandes use300  252 Feb 16  2021 .gitconfig
    -rw-------   1 mkandes use300    0 Feb 17  2022 .viminfo.tmp
    drwxr-xr-x   5 mkandes use300    6 Dec 12  2023 .ood_portal
    drwxr-xr-x  12 mkandes use300   12 Jul  3  2024 projects
    -rw-r--r--   1 mkandes use300  414 Sep 20  2024 .bash_profile
    drwxr-xr-x   2 mkandes use300    3 Oct  1  2024 .vim
    -rw-------   1 mkandes use300  177 Nov 18  2024 .sh_history
    drwx------   2 mkandes use300    7 Mar 14  2025 .ssh
    -rw-------   1 mkandes use300    0 Apr  8  2025 .history
    -rw-r--r--   1 mkandes use300 786M Jun  1 23:51 ilastik-1.4.1.post1-Linux.tar.bz2
    drwxr-xr-x  13 mkandes use300   16 Aug 10 14:14 data
    drwx------   3 mkandes use300    3 Aug 22 13:21 .singularity
    -rw-r--r--   1 mkandes use300 1.7K Oct 10 09:05 .bashrc
    -rw-r--r--   1 mkandes use300 1.2K Nov  3 10:25 .wget-hsts
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Desktop
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Downloads
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Templates
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Public
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Documents
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Music
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Pictures
    drwxr-xr-x   2 mkandes use300    2 Nov  3 12:25 Videos
    drwxr-xr-x   3 mkandes use300    3 Nov  3 13:14 .local
    drwxr-xr-x   6 mkandes use300    6 Nov  3 13:17 .cache
    drwxr-xr-x   3 mkandes use300    3 Nov  3 15:21 tmp
    drwx------   3 mkandes use300    3 Nov  5 13:30 .dbus
    -rw-------   1 mkandes use300    0 Nov  5 13:30 .ICEauthority
    drwxr-xr-x   7 mkandes use300    9 Nov 10 20:01 .config
    -rw-------   1 mkandes use300  305 Nov 13 11:37 .Xauthority
    drwxr-xr-x   2 mkandes use300    7 Nov 13 11:37 .vnc
    drwx------   3 mkandes use300    7 Nov 13 11:37 .gnupg
    drwxr-xr-x   2 mkandes use300    2 Nov 14 13:40 .ipython
    drwxr-xr-x   4 mkandes use300    4 Nov 14 13:41 .jupyter
    drwx------   3 mkandes use300    3 Nov 14 14:43 .nv
    -rw-------   1 mkandes use300 2.2K Nov 14 15:48 .python_history
    drwx------   2 mkandes use300    6 Nov 17 14:54 .galyleo
    drwxr-xr-x 109 mkandes use300  111 Nov 21 15:53 software
    -rw-------   1 mkandes use300  958 Nov 23 17:53 .lesshst
    -rw-------   1 mkandes use300  52K Nov 24 07:16 .viminfo
    drwxr-x---  29 mkandes use300   46 Nov 24 07:16 .
    -rw-------   1 mkandes use300 798K Nov 24 07:36 .bash_history
    drwxr-xr-x   3 mkandes use300   17 Nov 24 07:42 scripts
    drwxr-xr-x 417 root    root      0 Nov 24 07:42 ..
    [mkandes@login01 ~]$
    
  2. mkandes commented on Nov 24, 2025

    @mkandes
    MemberAuthor

    Seems to be limited to implicitly installed software applications + dependencies.

  3. mkandes commented on Dec 17, 2025

    @mkandes
    MemberAuthor

    After running a refresh, it still seems that the HIDDEN_AS_IMPLICIT are not hidden.

    [spack_cpu@exp-15-56 yamls]$ spack -d module lmod refresh --delete-tree -y
    ...
    ==> [2025-12-16-16:38:11.278550] 	HIDDEN_AS_IMPLICIT : zstd@1.5.5%gcc@8.5.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-zen/a6unagk
    ==> [2025-12-16-16:38:11.279725] 	HIDDEN_AS_IMPLICIT : zstd@1.5.5%gcc@13.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-zen2/el7dac3
    ==> [2025-12-16-16:38:11.279766] 	WRITE: zstd@1.5.5%gcc@13.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-zen2/el7dac3 [/cm/shared/apps/spack/0.21.2/cpu/dev/share/spack/lmod/linux-rocky8-x86_64/gcc/13.3.0/zstd/1.5.5-el7dac3-el7dac3.lua]
    ==> [2025-12-16-16:38:11.280503] 	HIDDEN_AS_IMPLICIT : zstd@1.5.5%gcc@13.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-zen2/el7dac3
    ==> [2025-12-16-16:38:11.293831] 	HIDDEN_AS_IMPLICIT : zstd@1.5.5%gcc@13.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-zen2/el7dac3
    ==> [2025-12-16-16:38:11.295033] 	HIDDEN_AS_IMPLICIT : zstd@1.5.5%gcc@13.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-zen2
    ...
    
    [mkandes_test@login01]~% . /cm/shared/apps/spack/0.21.2/cpu/dev/share/spack/setup-env.sh
    [mkandes_test@login01]~% module use /cm/shared/apps/spack/0.21.2/cpu/dev/share/spack/lmod/linux-rocky8-x86_64/Core
    [mkandes_test@login01]~% module avail
    
     /cm/shared/apps/spack/0.21.2/cpu/dev/share/spack/lmod/linux-rocky8-x86_64/Core 
       aocc/4.2.0-unqsjyx                    go/1.23.1-wmua5du
       aria2/1.37.0-x7zvjlo                  mercurial/6.8.1-6xah224
       bzip2/1.0.8-lpkmsaf                   openjdk/11.0.24_8-ptaqsmo
       cmake/3.27.7-ddtgwtz                  parallel/20240922-noglfjc
       curl/8.4.0-o2at4zg                    perl/5.38.2-rnajrbt
       entrezdirect/22.6.20240924-2hu7dm6    pigz/2.8-gmshkga
       gcc/13.3.0-t46rsdv                    rclone/1.68.1-k2idqyy
       gdb/14.2-4z3cgjb                      sqlite/3.43.2-wjd5v6s
       gh/2.57.0-ggsggei                     sratoolkit/3.1.1-wpdp5nr
       git-lfs/3.5.1-kbod4qa                 subversion/1.14.3-7vk5qba
       git/2.45.2-pxyqs55                    tar/1.34-2tp52li
    
    - /cm/shared/apps/spack/0.17.3/cpu/b/share/spack/lmod/linux-rocky8-x86_64/Core -
       anaconda3/2021.05/q4munrg             matlab/2022b/lefe4oq
       aocc/3.2.0/io3s466                    mercurial/5.8/qmgrjvl
       aria2/1.35.0/q32jtg2                  nvhpc/21.9/xxpthf5
       cmake/3.21.4/n5jtjsf                  parallel/20210922/sqru6rr
       entrezdirect/10.7.20190114/6pkkpx2    pigz/2.6/bgymyil
       gcc/10.2.0/npcyll4                    rclone/1.56.2/mldjorr
       gh/2.0.0/mkz3uxl                      sratoolkit/2.10.9/rn4humf
       git-lfs/2.11.0/kmruniy                subversion/1.14.0/qpzq6zs
    [mkandes_test@login01]~% module load gcc/13.3.0-t46rsdv     
    
    Inactive Modules:
      1) slurm
    
    [mkandes_test@login01]~% module list
    
    Currently Loaded Modules:
      1) shared                          6) mpfr/4.2.0-lp44usi-lp44usi
      2) cpu/0.17.3b               (c)   7) mpc/1.3.1-sw4uskz-sw4uskz
      3) sdsc/1.0                        8) zlib-ng/2.1.4-f3aqwo6-f3aqwo6
      4) DefaultModules                  9) zstd/1.5.5-a6unagk-a6unagk
      5) gmp/6.2.1-siwf7iu-siwf7iu      10) gcc/13.3.0-t46rsdv
    
      Where:
       c:  built natively for AMD Rome
    
    Inactive Modules:
      1) slurm
    
     
    
    [mkandes_test@login01]~%
    

    And, of course, the double appended hash remains as well.

  4. mkandes commented on Dec 17, 2025

    @mkandes
    MemberAuthor

    This is the current modules.yaml file. What happens if we revert to the prior '{name}/{version}/{hash:7}' format?

    modules:
      default:
        enable::
          - lmod
        lmod:
          core_compilers:
            - 'gcc@8.5.0'
          hierarchy:
            - mpi
          hash_length: 0
          hide_implicits: true
          naming_scheme: '{name}/{version}-{hash:7}'
          projections:
            all: '{name}/{version}-{hash:7}'
          all:
            autoload: 'direct'
            environment:
              set:
                '{name}_ROOT': '{prefix}'
            suffixes:
              '+complex': cmplx
              '+openmp': omp
              'threads=openmp': omp
              '+ilp64': i64
          intel:
            environment:
              set:
                INTEL_LICENSE_FILE: '40000@elprado.sdsc.edu:40200@elprado.sdsc.edu'
    
  5. mkandes commented on Dec 17, 2025

    @mkandes
    MemberAuthor

    Let's first try and do a more in-depth refresh ...

    spack clean --all
    spack reindex
    spack -d module lmod refresh --delete-tree -y
    

    No dice.

  6. mkandes commented on Dec 17, 2025

    @mkandes
    MemberAuthor

    Switching to / hash-format does not change anything.

    [mkandes_test@login01]~% module load gcc/13.3.0/
    [mkandes_test@login01]~% module list 
    
    Currently Loaded Modules:
      1) shared                          7) mpfr/4.2.0/lp44usi-lp44usi
      2) cpu/0.17.3b               (c)   8) mpc/1.3.1/sw4uskz-sw4uskz
      3) slurm/expanse/23.02.7           9) zlib-ng/2.1.4/f3aqwo6-f3aqwo6
      4) sdsc/1.0                       10) zstd/1.5.5/a6unagk-a6unagk
      5) DefaultModules                 11) gcc/13.3.0/t46rsdv
      6) gmp/6.2.1/siwf7iu-siwf7iu
    
      Where:
       c:  built natively for AMD Rome
    
     
    
    [mkandes_test@login01]~%
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions