-
Notifications
You must be signed in to change notification settings - Fork 763
HPCX: add configs to build OSS part of HPCX #7725
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 12 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
4c4a5b2
HPCX: add configs to build OSS part of HPCX
0f14bf4
HPCX: update configs to match HPCX v2.3
ea8246e
UCX: rename HPCX_UCX into UCX-hpcx
add5016
OMPI: rename HPCX_OMPI into OpenMPI-hpcx
6eab44f
UCX: upgrade to GCCcore 8.2.0
10f599e
OMPI: add preconfigure action
2e2d01b
HPCX: update toolchain and dependencies
59751ad
OMPI: fix version
c6c2363
OMPI: fix version
549b953
Address review comments
25c144e
UCX: download to unambiguous file name
d14d0ca
Address review comments
f862589
Use official OMPI release
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
18 changes: 18 additions & 0 deletions
18
easybuild/easyconfigs/h/HPCX/HPCX-2.3.0-GCC-8.2.0-2.31.1.eb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| easyblock = 'Bundle' | ||
|
|
||
| name = 'HPCX' | ||
| version = '2.3.0' | ||
|
|
||
| homepage = 'http://www.mellanox.com/page/products_dyn?product_family=189&mtag=hpc-x' | ||
| description = """The Mellanox HPC-X Toolkit | ||
| is a comprehensive MPI and SHMEM/PGAS software suite | ||
| for high performance computing environments""" | ||
|
|
||
| toolchain = {'name': 'GCC', 'version': '8.2.0-2.31.1'} | ||
|
|
||
| dependencies = [ | ||
| ('UCX', '1.5.0rc1', '-hpcx'), | ||
| ('OpenMPI', '4.0.x', '-hpcx') | ||
| ] | ||
|
|
||
| moduleclass = 'mpi' |
43 changes: 43 additions & 0 deletions
43
easybuild/easyconfigs/o/OpenMPI/OpenMPI-4.0.x-GCC-8.2.0-2.31.1-hpcx.eb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| easyblock = 'ConfigureMake' | ||
|
|
||
| name = 'OpenMPI' | ||
| version = '4.0.x' | ||
| versionsuffix = '-hpcx' | ||
|
|
||
| homepage = 'http://www.open-mpi.org/' | ||
| description = """Mellanox flavored Open MPI Project is an open source MPI-3 implementation.""" | ||
|
|
||
| toolchain = {'name': 'GCC', 'version': '8.2.0-2.31.1'} | ||
|
|
||
| sources = [{ | ||
| 'filename': 'openmpi-4.0.x.tar.gz', | ||
| 'git_config': { | ||
| 'url': 'https://github.com/open-mpi', | ||
| 'repo_name': 'ompi', | ||
| 'commit': '03cf3e4', | ||
| }, | ||
| }] | ||
| checksums = ['ef53605823b01499e09844a610614a555724603c72a1bb9a5ca59b35389f67ec'] | ||
|
|
||
| # needed for --with-verbs | ||
| osdependencies = [('libibverbs-dev', 'libibverbs-devel', 'rdma-core-devel')] | ||
|
|
||
| dependencies = [ | ||
| ('zlib', '1.2.11'), | ||
| ('UCX', '1.5.0rc1', '-hpcx') | ||
| ] | ||
|
|
||
| preconfigopts = "./autogen.pl && " | ||
| configopts = '--with-libevent=internal --enable-mpi1-compatibility ' | ||
| configopts = '--with-slurm --with-platform=contrib/platform/mellanox/optimized ' | ||
| configopts += '--with-ucx=$EBROOTUCX ' | ||
|
|
||
| libs = ["mpi_mpifh", "mpi", "ompitrace", "open-pal", "open-rte"] | ||
| sanity_check_paths = { | ||
| 'files': ["bin/%s" % binfile for binfile in ["ompi_info", "opal_wrapper", "orterun"]] + | ||
| ["lib/lib%s.%s" % (libfile, SHLIB_EXT) for libfile in libs] + | ||
| ["include/%s.h" % x for x in ["mpi-ext", "mpif-config", "mpif", "mpi", "mpi_portable_platform"]], | ||
| 'dirs': [], | ||
| } | ||
|
|
||
| moduleclass = 'mpi' | ||
47 changes: 47 additions & 0 deletions
47
easybuild/easyconfigs/u/UCX/UCX-1.5.0rc1-GCCcore-8.2.0-hpcx.eb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Note: | ||
| # This is an easyconfig file for EasyBuild, see https://github.com/easybuilders/easybuild | ||
| easyblock = 'ConfigureMake' | ||
|
|
||
| name = 'UCX' | ||
| version = '1.5.0rc1' | ||
| versionsuffix = '-hpcx' | ||
|
|
||
| homepage = 'http://www.openucx.org/' | ||
| description = """Unified Communication X | ||
| An open-source production grade communication framework for data centric | ||
| and high-performance applications | ||
| """ | ||
|
|
||
| toolchain = {'name': 'GCCcore', 'version': '8.2.0'} | ||
| toolchainopts = {'pic': True} | ||
|
|
||
| source_urls = ['https://github.com/openucx/ucx/releases/download/v1.5.0-rc1'] | ||
| sources = [{ | ||
| 'filename': 'ucx-1.5.0rc1.tar.gz', | ||
| 'download_filename': 'ucx-1.5.0.tar.gz' | ||
| }] | ||
| checksums = ['ae5dfa59c54627a11d8cb1c7fdd0b0319e5e8633a6aebd42a07e54a6a4a32181'] | ||
|
|
||
| builddependencies = [ | ||
| ('binutils', '2.31.1'), | ||
| ('pkg-config', '0.29.2'), | ||
| ] | ||
|
|
||
| osdependencies = [('libibverbs-dev', 'libibverbs-devel', 'rdma-core-devel')] | ||
|
|
||
| dependencies = [ | ||
| ('numactl', '2.0.12'), | ||
| ] | ||
|
|
||
| configopts = '--enable-optimizations --enable-optimizations --disable-logging ' | ||
| configopts += '--disable-debug --disable-assertions --disable-params-check ' | ||
| configopts += '--without-java --disable-doxygen-doc ' | ||
|
|
||
| buildopts = 'V=1' | ||
|
|
||
| sanity_check_paths = { | ||
| 'files': ['bin/ucx_info', 'bin/ucx_perftest', 'bin/ucx_read_profile'], | ||
| 'dirs': ['include', 'lib', 'share'] | ||
| } | ||
|
|
||
| moduleclass = 'mpi' |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This checksum is never going to be the same due to how it is created.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? Is there any way to fix that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
github produces the tar file on demand by first making a checkout of that commit causing directories to have timestamps that differs every time.
Don't think there is a way to fix that.
@boegel ??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@boegel could you please advise?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for not coming back to this...
For now, there's no way to reliable checksum tarballs that were created via
git_config, see also discussion in easybuilders/easybuild-framework#2727