Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ZEN-38311: PKG/SPEC - expanse/0.17.3/cpu/b - Update STAR to v2.7.11b #129

Open
mkandes opened this issue Sep 20, 2024 · 11 comments
Open

ZEN-38311: PKG/SPEC - expanse/0.17.3/cpu/b - Update STAR to v2.7.11b #129

mkandes opened this issue Sep 20, 2024 · 11 comments

Comments

@mkandes
Copy link
Member

mkandes commented Sep 20, 2024

User requested [1] if we could update STAR [2] on Expanse. An initial look at STAR last week indicated that it should be possible to deploy if via Spack v0.17.3 by simply updating the package.py file to include the new version [3] since there are almost no dependencies or variants to consider.

[1] https://sdsc.zendesk.com/agent/tickets/38311
[2] https://github.com/alexdobin/STAR
[3]

[spack_cpu@login02 ~]$ spack info star
Package:   star

Description:
    STAR is an ultrafast universal RNA-seq aligner.

Homepage: https://github.com/alexdobin/STAR

Externally Detectable: 
    False

Tags: 
    None

Preferred version:  
    2.7.10a    https://github.com/alexdobin/STAR/archive/2.7.10a.tar.gz

Safe versions:  
    2.7.10a    https://github.com/alexdobin/STAR/archive/2.7.10a.tar.gz
    2.7.9a     https://github.com/alexdobin/STAR/archive/2.7.9a.tar.gz
    2.7.6a     https://github.com/alexdobin/STAR/archive/2.7.6a.tar.gz
    2.7.0e     https://github.com/alexdobin/STAR/archive/2.7.0e.tar.gz
    2.7.0d     https://github.com/alexdobin/STAR/archive/2.7.0d.tar.gz
    2.6.1b     https://github.com/alexdobin/STAR/archive/2.6.1b.tar.gz
    2.6.1a     https://github.com/alexdobin/STAR/archive/2.6.1a.tar.gz
    2.6.0c     https://github.com/alexdobin/STAR/archive/2.6.0c.tar.gz
    2.6.0b     https://github.com/alexdobin/STAR/archive/2.6.0b.tar.gz
    2.6.0a     https://github.com/alexdobin/STAR/archive/2.6.0a.tar.gz
    2.5.4b     https://github.com/alexdobin/STAR/archive/2.5.4b.tar.gz
    2.5.4a     https://github.com/alexdobin/STAR/archive/2.5.4a.tar.gz
    2.5.3a     https://github.com/alexdobin/STAR/archive/2.5.3a.tar.gz
    2.5.2b     https://github.com/alexdobin/STAR/archive/2.5.2b.tar.gz
    2.5.2a     https://github.com/alexdobin/STAR/archive/2.5.2a.tar.gz
    2.4.2a     https://github.com/alexdobin/STAR/archive/STAR_2.4.2a.tar.gz

Deprecated versions:  
    None

Variants:
    None

Installation Phases:
    install

Build Dependencies:
    zlib

Link Dependencies:
    zlib

Run Dependencies:
    None

Virtual Packages: 
    None

[spack_cpu@login02 ~]$
@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Clone fork and create new feature branch.

mkandes@hardtack:~$ git clone https://github.com/mkandes/spack.git
Cloning into 'spack'...
remote: Enumerating objects: 568174, done.
remote: Counting objects: 100% (483/483), done.
remote: Compressing objects: 100% (212/212), done.
remote: Total 568174 (delta 191), reused 483 (delta 191), pack-reused 567691 (from 1)
Receiving objects: 100% (568174/568174), 246.30 MiB | 47.50 MiB/s, done.
Resolving deltas: 100% (259511/259511), done.
Updating files: 100% (12439/12439), done.
mkandes@hardtack:~$ cd spack/
mkandes@hardtack:~/spack$ git branch
* sdsc-0.17.3
mkandes@hardtack:~/spack$ git log -1
commit 1189ec74627ea872a5b5ed918cff7cf1368722a1 (HEAD -> sdsc-0.17.3, origin/sdsc-0.17.3, origin/HEAD)
Author: Marty Kandes <[email protected]>
Date:   Tue Aug 27 16:22:24 2024 -0700

    Deploy [email protected] % [email protected] to exp/0.17.3/cpu/b
mkandes@hardtack:~/spack$ git branch -a
* sdsc-0.17.3
  remotes/origin/HEAD -> origin/sdsc-0.17.3
  remotes/origin/develop
  remotes/origin/sdsc-0.17.3
  remotes/origin/sdsc-0.17.3-gh-124-inst-exp-cpu-dev
  remotes/origin/sdsc-0.17.3-gh-124-inst-exp-gpu-dev
  remotes/origin/sdsc-0.17.3-gh-125-spec-exp-cpu-pgi-netcdf
  remotes/origin/sdsc-0.17.3-gh-127-spec-exp-cpu-intel-tbb
mkandes@hardtack:~/spack$ git checkout -b sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star
Switched to a new branch 'sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star'
mkandes@hardtack:~/spack$ git branch
  sdsc-0.17.3
* sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star
mkandes@hardtack:~/spack$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

STAR is already a custom package.py file in the SDSC package repo. I do not recall what the custom changes were here.

mkandes@hardtack:~/spack$ cd var/spack/repos/
mkandes@hardtack:~/spack/var/spack/repos$ ls
builtin  builtin.mock  sdsc  tscc  tutorial
mkandes@hardtack:~/spack/var/spack/repos$ cd sdsc/
mkandes@hardtack:~/spack/var/spack/repos/sdsc$ ls
packages  repo.yaml
mkandes@hardtack:~/spack/var/spack/repos/sdsc$ cd packages/
mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages$ ls
amber         aocc         intel      openmpi           relion   vmd
amdblis       aocl-sparse  libbeagle  orca              spark    wannier90
amdfftw       gamess       matlab     perl              star
amdlibflame   gaussian     mbedtls    py-pyscf          tcl
amdlibm       gdbm         mvapich2   qchem             trinity
amdscalapack  gromacs      nwchem     quantum-espresso  vasp6
mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages$ cd star/
mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages/star$ ls
package.py
mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages/star$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

It looks like the only change was similar to this one. Include new version.

mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages/star$ diff package.py ../../../builtin/packages/star/package.py 
15,16d14
<     version('2.7.10a', sha256='af0df8fdc0e7a539b3ec6665dce9ac55c33598dfbc74d24df9dae7a309b0426a')
<     version('2.7.9a', sha256='ff52c9d6daaa9fb7261efa3aa49ef6ce5262aa089b0762a3cbc751e81321050e')
mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages/star$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Added latest versions of STAR to updated package.py file.

mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages/star$ git diff package.py
diff --git a/var/spack/repos/sdsc/packages/star/package.py b/var/spack/repos/sdsc/packages/star/package.py
index ccd4268ab7..3f83fd1f49 100644
--- a/var/spack/repos/sdsc/packages/star/package.py
+++ b/var/spack/repos/sdsc/packages/star/package.py
@@ -12,6 +12,10 @@ class Star(Package):
     homepage = "https://github.com/alexdobin/STAR"
     url      = "https://github.com/alexdobin/STAR/archive/2.7.6a.tar.gz"
 
+
+    version('2.7.11b', sha256='3f65305e4112bd154c7e22b333dcdaafc681f4a895048fa30fa7ae56cac408e7')
+    version('2.7.11a', sha256='542457b1a4fee73f27a581b1776e9f73ad2b4d7e790388b6dc71147bd039f99a')
+    version('2.7.10b', sha256='0d1b71de6c5be1c5d90b32130d2abcd5785a4fc7c1e9bf19cc391947f2dc46e5')
     version('2.7.10a', sha256='af0df8fdc0e7a539b3ec6665dce9ac55c33598dfbc74d24df9dae7a309b0426a')
     version('2.7.9a', sha256='ff52c9d6daaa9fb7261efa3aa49ef6ce5262aa089b0762a3cbc751e81321050e')
     version('2.7.6a', sha256='9320797c604673debea0fe8f2e3762db364915cc59755de1a0d87c8018f97d51')
mkandes@hardtack:~/spack/var/spack/repos/sdsc/packages/star$

mkandes added a commit to mkandes/spack that referenced this issue Sep 20, 2024
@mkandes
Update package.py for STAR within custom sdsc package repo
4effd11 
See sdsc#129
@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Added [email protected] % [email protected] spec build script to exp/0.17.3/cpu/b

mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$ cp [email protected] [email protected] 
mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$ vi [email protected] 
mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$ git add [email protected] 
mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$ git commit
[sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star 799e7ded9b] Add [email protected] % [email protected] to exp/0.17.3/cpu/b
 1 file changed, 67 insertions(+)
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]/[email protected]
mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$ git push
Username for 'https://github.com': mkandes
Password for 'https://[email protected]': 
Enumerating objects: 22, done.
Counting objects: 100% (22/22), done.
Delta compression using up to 8 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (12/12), 1.62 KiB | 830.00 KiB/s, done.
Total 12 (delta 4), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To https://github.com/mkandes/spack.git
   4effd11498..799e7ded9b  sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star -> sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star
mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$ git log -2
commit 799e7ded9b2a13a52e708f8010d75deee4f1d4a6 (HEAD -> sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star, origin/sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star)
Author: Marty Kandes <[email protected]>
Date:   Fri Sep 20 15:51:47 2024 -0700

    Add [email protected] % [email protected] to exp/0.17.3/cpu/b

commit 4effd114980758dd367345ff42d433bb94b3d122
Author: Marty Kandes <[email protected]>
Date:   Fri Sep 20 15:44:57 2024 -0700

    Update package.py for STAR within custom sdsc package repo
    
    See https://github.com/sdsc/spack/issues/129
mkandes@hardtack:~/spack/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]$

@mkandes mkandes mentioned this issue Sep 20, 2024
Merged
@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Pull request created and merged. #130

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Pull changes to production instance.

[spack_cpu@exp-15-56 b]$ git pull
remote: Enumerating objects: 22, done.
remote: Counting objects: 100% (22/22), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 22 (delta 9), reused 21 (delta 9), pack-reused 0 (from 0)
Unpacking objects: 100% (22/22), 3.34 KiB | 103.00 KiB/s, done.
From https://github.com/sdsc/spack
   1189ec7462..6065106649  sdsc-0.17.3 -> origin/sdsc-0.17.3
Updating 1189ec7462..6065106649
Fast-forward
 .../0.17.3/cpu/b/specs/[email protected]/[email protected]  | 67 ++++++++++++++++++++++
 var/spack/repos/sdsc/packages/star/package.py      |  4 ++
 2 files changed, 71 insertions(+)
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/[email protected]/[email protected]
[spack_cpu@exp-15-56 b]$ git log -2
commit 6065106649234486cad73e7ef30fe095de5b414d (HEAD -> sdsc-0.17.3, origin/sdsc-0.17.3, origin/HEAD)
Merge: 1189ec7462 799e7ded9b
Author: Marty Kandes, Ph.D. <[email protected]>
Date:   Fri Sep 20 15:57:57 2024 -0700

    Merge pull request #130 from mkandes/sdsc-0.17.3-gh-129-pkg-spec-exp-cpu-star
    
    Add [email protected] % [email protected] to expanse/0.17.3/cpu/b

commit 799e7ded9b2a13a52e708f8010d75deee4f1d4a6
Author: Marty Kandes <[email protected]>
Date:   Fri Sep 20 15:51:47 2024 -0700

    Add [email protected] % [email protected] to exp/0.17.3/cpu/b
[spack_cpu@exp-15-56 b]$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024 •
edited
Loading

Deploying package spec.

[spack_cpu@exp-15-56 [email protected]]$ sbatch [email protected] 
Submitted batch job 34015247
[spack_cpu@exp-15-56 [email protected]]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          34015247 ind-share [email protected] spack_cp  R       0:09      1 exp-15-56
          34015042 ind-share     bash spack_cp  R      10:09      1 exp-15-56
[spack_cpu@exp-15-56 [email protected]]$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Build looks successful.

==> Fetching https://mirror.spack.io/_source-cache/archive/3f/3f65305e4112bd154c7e22b333dcdaafc681f4a895048fa30fa7ae56cac408e7.tar.gz
==> No patches needed for star
==> star: Executing phase: 'install'
==> star: Successfully installed star-2.7.11b-c27utuxywta6qft47zzgghmxj3wnt7ba
  Fetch: 1.00s.  Build: 51.23s.  Total: 52.22s.
[+] /cm/shared/apps/spack/0.17.3/cpu/b/opt/spack/linux-rocky8-zen2/gcc-10.2.0/star-2.7.11b-c27utuxywta6qft47zzgghmxj3wnt7ba
real 54.73
user 170.49
sys 254.07

[spack_cpu@exp-15-56 [email protected]]$ spack find -lvd [email protected]
==> 1 installed package
-- linux-rocky8-zen2 / [email protected] -------------------------------
c27utux [email protected]
ws4iari     [email protected]+optimize+pic+shared

[spack_cpu@exp-15-56 [email protected]]$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

Deployment complete after module refresh. User notified for testing.

[mkandes@login02 ~]$ module load gcc/10.2.0
[mkandes@login02 ~]$ module load star/2.7.11b
[mkandes@login02 ~]$ which start
start-statd  startx       
[mkandes@login02 ~]$ which STAR
/cm/shared/apps/spack/0.17.3/cpu/b/opt/spack/linux-rocky8-zen2/gcc-10.2.0/star-2.7.11b-c27utuxywta6qft47zzgghmxj3wnt7ba/bin/STAR
[mkandes@login02 ~]$

@mkandes
Copy link
Member Author

mkandes commented Sep 20, 2024

b62c4b1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mkandes