HPCX: add configs to build OSS part of HPCX #7725

amaslenn · 2019-02-25T10:52:09Z

No description provided.

bartoldeman · 2019-02-25T15:11:48Z

Note that UCX from HPCX 2.3 is exactly the same as UCX 1.5.0rc1 (git checkout 02078b9), and OpenMPI 4.0.0 corresponds to v4.0.0-34-g03cf3e4, ie. git checkout 03cf3e4, which is a checkout of the v4.0.x branch of OpenMPI.

My personal opinion is that if you compile your own OpenMPI, then HPCX is not so useful except exactly the proprietary components (hcoll and perhaps sharp in particular). Especially now since UCX 1.5.0 without rc has been released already. OMPI 4.0.1 should be coming soon too I guess.

amaslenn · 2019-02-28T08:59:09Z

@bartoldeman I've update configs to match HPCX v2.3 exactly. Currently, the idea is to have open source parts of HPCX available through EasyBuild system. Once HPCX v2.4 available, I'll create an update for EB configs.

Have a question regarding CI: I checked my configs with --robot=<scripts path> since they are not yet in repo. Is there a way to do it in CI? Or is there another option to pass verification?

amaslenn · 2019-03-11T07:34:22Z

@bartoldeman could you please take a look?

akesandgren · 2019-03-19T07:41:36Z

easybuild/easyconfigs/h/HPCX/HPCX_UCX-1.5.0rc1-foss-2018b.eb

UCX belongs at the GCCcore level, and it's better to use official releases of it instead of release candidates. See #7924

boegel · 2019-03-19T07:43:54Z

@amaslenn The problem is that we require that easyconfig files are organized in subdirectories based on software name.

So, HPCX_OMPI-4.0.0-foss-2018b.eb should be located in the h/HPCX_OMPI subdirectory; similar for HPCX_UCX.

It may be nicer to use a versionsuffix here instead, to keep things together.

That would involve renaming the HPCX_OMPI-4.0.0-foss-2018b.eb easyconfig file to HPCX-4.0.0-foss-2018b-ompi.eb, changing the software name to HPCX (for all 3 easyconfigs), and using versionsuffix = '-ompi' (and similar for HPCX_UCX).

I think the latter makes more sense, maybe?

akesandgren · 2019-03-19T07:51:37Z

easybuild/easyconfigs/h/HPCX/HPCX_OMPI-4.0.x-foss-2018b.eb

If you're picking this up from OpenMPI directly then it is not the Mellanox version and it is just a normal OpenMPI
And OpenMPI belongs at the GCC level not foss.

akesandgren · 2019-03-19T07:57:45Z

This basically looks redundant since it's just a tweaked build of OpenMPI with UCX.

amaslenn · 2019-03-21T04:56:05Z

@boegel @akesandgren I updated configs, now both UCX and OMPI are present in standard locations, but I'm still getting "missing dependencies" issue. Could you please help me understand why?

I also applied all the changes you've suggested. Could you please review and let me know if I missed something?

akesandgren · 2019-03-21T06:23:42Z

easybuild/easyconfigs/h/HPCX/HPCX-2.3.0-GCC-8.2.0-2.31.1.eb

+
+dependencies = [
+    ('UCX', '1.5.0rc1', '-hpcx', ('GCCcore', '8.2.0')),
+    ('OpenMPI', '4.0.x', '-hpcx', ('GCC', '8.2.0-2.31.1'))


You're still requesting 4.0.x version, but the OpenMPI easyconfig below is 4.0.0. You have to decide if you want a 4.0.0 version or the "4.0.x" version. Do not call the OpenMPI easyconfig 4.0.0 if it is not using the official 4.0.0 release.

Argh, missed that. Thanks! Version renamed to 4.0.x. Is OK?

akesandgren · 2019-03-21T06:24:43Z

easybuild/easyconfigs/o/OpenMPI/OpenMPI-4.0.0-GCC-8.2.0-2.31.1-hpcx.eb

+    'git_config': {
+        'url': 'https://github.com/open-mpi',
+        'repo_name': 'ompi',
+        'commit': '03cf3e4',


And repeating what i said in the other comment. Do not set the version of this OpenMPI build to 4.0.0 if it is not using the official 4.0.0 release from OpenMPI!

akesandgren · 2019-03-21T09:07:03Z

easybuild/easyconfigs/h/HPCX/HPCX-2.3.0-GCC-8.2.0-2.31.1.eb

+toolchain = {'name': 'GCC', 'version': '8.2.0-2.31.1'}
+
+dependencies = [
+    ('UCX', '1.5.0rc1', '-hpcx', ('GCCcore', '8.2.0')),


You only need
('UCX', '1.5.0rc1', '-hpcx'),
('OpenMPI', '4.0.x', '-hpcx'),

It will find the correct UCX, OpenMPI version by itself.

akesandgren · 2019-03-21T09:07:36Z

easybuild/easyconfigs/o/OpenMPI/OpenMPI-4.0.x-GCC-8.2.0-2.31.1-hpcx.eb

+
+dependencies = [
+    ('zlib', '1.2.11'),
+    ('UCX', '1.5.0rc1', '-hpcx', ('GCCcore', '8.2.0'))


Same here, just ('UCX', '1.5.0rc1', '-hpcx'),

amaslenn · 2019-03-21T12:31:00Z

@akesandgren looks good?

akesandgren

LGTM

akesandgren · 2019-03-21T12:59:47Z

Test report by @akesandgren
FAILED
Build succeeded for 0 out of 3 (3 easyconfigs in this PR)
b-an03.hpc2n.umu.se - Linux ubuntu 16.04, Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, Python 2.7.12
See https://gist.github.com/c585b499d62f0ac47f169fb3688e8f95 for a full test report.

akesandgren · 2019-03-21T13:06:57Z

easybuild/easyconfigs/u/UCX/UCX-1.5.0rc1-GCCcore-8.2.0-hpcx.eb

+toolchainopts = {'pic': True}
+
+source_urls = ['https://github.com/openucx/ucx/releases/download/v1.5.0-rc1']
+sources = ['ucx-1.5.0.tar.gz']


Hmmmm, important note here, since you're pulling down 1.5.0-rc1 the source should file not be called ucx-1.5.0.tar.gz.
You need to do it like this or there will be checksum conflicts for people who already have the real 1.5.0 downloaded.

sources = [ {'filename': 'ucx-1.5.0-rc1.tar.gz', 'download_filename', 'ucx-1.5.0.tar.gz'} ]

amaslenn · 2019-03-26T04:17:29Z

@akesandgren could you please re-run tests?

akesandgren · 2019-03-26T06:21:37Z

easybuild/easyconfigs/u/UCX/UCX-1.5.0rc1-GCCcore-8.2.0-hpcx.eb

+    'filename': 'ucx-1.5.0rc1.tar.gz',
+    'download_filename': 'ucx-1.5.0.tar.gz'
+}]
+source_urls = ['https://github.com/openucx/ucx/releases/download/v1.5.0-rc1']


Source_urls should come before sources line

akesandgren · 2019-03-26T06:32:53Z

Test report by @akesandgren
FAILED
Build succeeded for 1 out of 3 (3 easyconfigs in this PR)
b-an03.hpc2n.umu.se - Linux ubuntu 16.04, Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, Python 2.7.12
See https://gist.github.com/4859b6d7e140cea68d7b1b08614781a5 for a full test report.

akesandgren · 2019-03-26T07:19:30Z

Test report by @akesandgren
FAILED
Build succeeded for 1 out of 3 (3 easyconfigs in this PR)
b-an03.hpc2n.umu.se - Linux ubuntu 16.04, Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, Python 2.7.12
See https://gist.github.com/03fdc914f7b6386f19b7a9ee298ac7c7 for a full test report.

akesandgren · 2019-03-26T08:01:34Z

easybuild/easyconfigs/o/OpenMPI/OpenMPI-4.0.x-GCC-8.2.0-2.31.1-hpcx.eb

+        'commit': '03cf3e4',
+    },
+}]
+checksums = ['ef53605823b01499e09844a610614a555724603c72a1bb9a5ca59b35389f67ec']


This checksum is never going to be the same due to how it is created.

Why? Is there any way to fix that?

github produces the tar file on demand by first making a checkout of that commit causing directories to have timestamps that differs every time.
Don't think there is a way to fix that.

@boegel ??

@boegel could you please advise?

Sorry for not coming back to this...

For now, there's no way to reliable checksum tarballs that were created via git_config, see also discussion in easybuilders/easybuild-framework#2727

amaslenn · 2019-03-31T10:02:44Z

@akesandgren I switched to use official OMPI release. Could you please review/test?

akesandgren · 2019-03-31T11:41:36Z

Test report by @akesandgren
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in this PR)
b-an03.hpc2n.umu.se - Linux ubuntu 16.04, Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, Python 2.7.12
See https://gist.github.com/314c2f3576b2bd975a0883247e63574c for a full test report.

akesandgren

LGTM

akesandgren · 2019-04-01T11:25:46Z

Going in, thanks @amaslenn!

boegel · 2019-04-02T05:53:30Z

Test report by @boegel
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in this PR)
node2416.golett.os - Linux centos linux 7.6.1810, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, Python 2.7.5
See https://gist.github.com/89d87b5e31456afe76e6ff97a31f6796 for a full test report.

amaslenn force-pushed the hpcx branch from c84e1aa to 939c228 Compare February 28, 2019 09:06

boegel added the new label Mar 19, 2019

akesandgren reviewed Mar 19, 2019

View reviewed changes

Andrey Maslennikov added 7 commits March 20, 2019 04:22

HPCX: add configs to build OSS part of HPCX

4c4a5b2

HPCX: update configs to match HPCX v2.3

0f14bf4

UCX: rename HPCX_UCX into UCX-hpcx

ea8246e

OMPI: rename HPCX_OMPI into OpenMPI-hpcx

add5016

UCX: upgrade to GCCcore 8.2.0

6eab44f

OMPI: add preconfigure action

10f599e

HPCX: update toolchain and dependencies

2e2d01b

amaslenn force-pushed the hpcx branch from 939c228 to 2e2d01b Compare March 20, 2019 09:56

OMPI: fix version

59751ad

akesandgren reviewed Mar 21, 2019

View reviewed changes

OMPI: fix version

c6c2363

akesandgren reviewed Mar 21, 2019

View reviewed changes

Address review comments

549b953

akesandgren approved these changes Mar 21, 2019

View reviewed changes

akesandgren added this to the 3.x milestone Mar 21, 2019

akesandgren requested changes Mar 21, 2019

View reviewed changes

UCX: download to unambiguous file name

25c144e

akesandgren requested changes Mar 26, 2019

View reviewed changes

Address review comments

d14d0ca

akesandgren reviewed Mar 26, 2019

View reviewed changes

Use official OMPI release

f862589

amaslenn force-pushed the hpcx branch from 258e5d8 to f862589 Compare March 31, 2019 10:28

akesandgren reviewed Apr 1, 2019

View reviewed changes

akesandgren approved these changes Apr 1, 2019

View reviewed changes

akesandgren merged commit 11db044 into easybuilders:develop Apr 1, 2019

amaslenn deleted the hpcx branch April 2, 2019 04:59

easybuilders deleted a comment from boegelbot Apr 2, 2019

boegel modified the milestones: 3.x, 3.9.0 Apr 2, 2019

HPCX: add configs to build OSS part of HPCX #7725

HPCX: add configs to build OSS part of HPCX #7725

Uh oh!

Conversation

amaslenn commented Feb 25, 2019

Uh oh!

bartoldeman commented Feb 25, 2019

Uh oh!

amaslenn commented Feb 28, 2019

Uh oh!

amaslenn commented Mar 11, 2019

Uh oh!

akesandgren Mar 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

boegel commented Mar 19, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akesandgren commented Mar 19, 2019

Uh oh!

amaslenn commented Mar 21, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amaslenn commented Mar 21, 2019

Uh oh!

akesandgren left a comment

Choose a reason for hiding this comment

Uh oh!

akesandgren commented Mar 21, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amaslenn commented Mar 26, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akesandgren commented Mar 26, 2019

Uh oh!

akesandgren commented Mar 26, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amaslenn commented Mar 31, 2019

Uh oh!

akesandgren commented Mar 31, 2019

Uh oh!

akesandgren left a comment

Choose a reason for hiding this comment

Uh oh!

akesandgren commented Apr 1, 2019

Uh oh!

boegel commented Apr 2, 2019

Uh oh!

akesandgren Mar 19, 2019 •

edited

Loading